o1-Coder: an o1 Replication for Coding
The technical report introduces O1-CODER, an attempt to replicate OpenAI's o1
model with a focus on coding tasks. It integrates reinforcement learning (RL)
and Monte Carlo Tree Search (MCTS) to enhance the model's System-2 thinking
capabilities. The framework includes training a Test Case Generator (TCG) for
standardized code testing, using MCTS to generate code data with reasoning
processes, and iteratively fine-tuning the policy model to initially produce
pseudocode, followed by the generation of the full code. The report also
addresses the opportunities and challenges in deploying o1-like models in
real-world applications, suggesting transitioning to the System-2 paradigm and
highlighting the imperative for environment state updates. Updated model
progress and experimental results will be reported in subsequent versions. All
source code, curated datasets, as well as the derived models will be disclosed
at https://github.com/ADaM-BJTU/O1-CODER .
674e99013209f1fbb76bbec6
Yuxiang Zhang
admin_assigned
2024-12-05T09:49:52.207Z
645b4a2978730bcc103dfe4d
/avatars/de544de899897fd0a83506ff287123bc.svg
Yuxiang Zhang
user
TokerZ
674e99013209f1fbb76bbec7
Shangxi Wu
admin_assigned
2024-12-03T15:16:33.713Z
665ace40ddba0f825f3a4941
/avatars/ed7e209e14b5ae495aa7f4baec980b38.svg
Shangxi Wu
user
KirinNg
674e99013209f1fbb76bbec8
Yuqi Yang
admin_assigned
2024-12-03T15:16:27.807Z
65b5b586d2ae98fd67c76d6b
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/RR_mNkUAG4zP902WLgeYv.jpeg
yuqi yang
user
tzteyang
674e99013209f1fbb76bbec9
Jiangming Shu
admin_assigned
2024-12-03T15:16:20.077Z
6540f059733c1ce6a6fcca88
/avatars/b8fa6b383caeb1fe79e163bac90d5cba.svg
ShuJiangMing
user
SanJiaoMao
674e99013209f1fbb76bbeca
Jinlin Xiao
674e99013209f1fbb76bbecb
Chao Kong
674e99013209f1fbb76bbecc
Jitao Sang
claimed_verified
2024-12-04T09:46:19.856Z
6494457c6339264dd78bcb95
/avatars/d87842251f1a43f50cc827f0e2a995ee.svg
sdzy
user
sdzy
o1-Coder: an o1 Replication for Coding
The technical report introduces O1-CODER, an attempt to replicate OpenAI's o1
model with a focus on coding tasks. It integrates reinforcement learning (RL)
and Monte Carlo Tree Search (MCTS) to enhance the model's System-2 thinking
capabilities. The framework includes training a Test Case Generator (TCG) for
standardized code testing, using MCTS to generate code data with reasoning
processes, and iteratively fine-tuning the policy model to initially produce
pseudocode, followed by the generation of the full code. The report also
addresses the opportunities and challenges in deploying o1-like models in
real-world applications, suggesting transitioning to the System-2 paradigm and
highlighting the imperative for environment state updates. Updated model
progress and experimental results will be reported in subsequent versions. All
source code, curated datasets, as well as the derived models will be disclosed
at https://github.com/ADaM-BJTU/O1-CODER .
O1-CODER integrates reinforcement learning and Monte Carlo Tree Search to enhance coding capabilities, focusing on pseudocode and full code generation through iterative fine-tuning and standardized code testing.
reinforcement learning Monte Carlo Tree Search Test Case Generator policy model pseudocode full code generation iterative fine-tuning standardized code testing