CoDA: Coding LM via Diffusion Adaptation
Diffusion language models promise bidirectional context and infilling
capabilities that autoregressive coders lack, yet practical systems remain
heavyweight. We introduce CoDA, a 1.7B-parameter diffusion coder trained on TPU
with a fully open-source training pipeline. CoDA pairs large-scale diffusion
pre-training with code-centric mid-training and instruction tuning, enabling
confidence-guided sampling that keeps inference latency competitive. On
Humaneval, MBPP, and EvalPlus, CoDA-1.7B-Instruct matches or surpasses
diffusion models up to 7B parameters. Our release includes model checkpoints,
evaluation harnesses, and TPU training pipelines to accelerate research on
lightweight diffusion-based coding assistants.
68e4787be4e093a7044e4ceb
Haolin Chen
claimed_verified
2025-10-10T09:38:51.604Z
65e0c21ebd0c59bbf271004c
/avatars/3210f41fd433d798fd7857c943288625.svg
Haolin Chen
user
hlnchen
68e4787be4e093a7044e4cec
Shiyu Wang
68e4787be4e093a7044e4ced
Can Qin
claimed_verified
2025-10-10T09:38:42.125Z
66a9bdfc35c89692442ba4b7
/avatars/ad87d1d2d81775eeb0920bf3ebe08cc2.svg
Can Qin
user
canqin001
68e4787be4e093a7044e4cee
Bo Pang
68e4787be4e093a7044e4cef
Zuxin Liu
68e4787be4e093a7044e4cf0
Jielin Qiu
68e4787be4e093a7044e4cf1
Jianguo Zhang
68e4787be4e093a7044e4cf2
Yingbo Zhou
68e4787be4e093a7044e4cf3
Zeyuan Chen
68e4787be4e093a7044e4cf4
Ran Xu
68e4787be4e093a7044e4cf5
Shelby Heinecke
68e4787be4e093a7044e4cf6
Silvio Savarese
68e4787be4e093a7044e4cf7
Caiming Xiong
68e4787be4e093a7044e4cf8
Huan Wang
68e4787be4e093a7044e4cf9
Weiran Yao
claimed_verified
2025-10-08T08:03:10.713Z
661573234c2f29635e93bb71
/avatars/fba95e382454485766b6349d6281b715.svg
Weiran Yao
user
weiranyao
CoDA: Coding LM via Diffusion Adaptation
Diffusion language models promise bidirectional context and infilling
capabilities that autoregressive coders lack, yet practical systems remain
heavyweight. We introduce CoDA, a 1.7B-parameter diffusion coder trained on TPU
with a fully open-source training pipeline. CoDA pairs large-scale diffusion
pre-training with code-centric mid-training and instruction tuning, enabling
confidence-guided sampling that keeps inference latency competitive. On
Humaneval, MBPP, and EvalPlus, CoDA-1.7B-Instruct matches or surpasses
diffusion models up to 7B parameters. Our release includes model checkpoints,
evaluation harnesses, and TPU training pipelines to accelerate research on
lightweight diffusion-based coding assistants.
CoDA, a 1.7B-parameter diffusion coder, achieves competitive performance with smaller models through confidence-guided sampling and is released with open-source tools.
diffusion language models bidirectional context infilling capabilities autoregressive coders diffusion coder large-scale diffusion pre-training code-centric mid-training instruction tuning confidence-guided sampling inference latency Humaneval MBPP EvalPlus diffusion-based coding assistants