SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open
  Software Evolution
The recent DeepSeek-R1 release has demonstrated the immense potential of
reinforcement learning (RL) in enhancing the general reasoning capabilities of
large language models (LLMs). While DeepSeek-R1 and other follow-up work
primarily focus on applying RL to competitive coding and math problems, this
paper introduces SWE-RL, the first approach to scale RL-based LLM reasoning for
real-world software engineering. Leveraging a lightweight rule-based reward
(e.g., the similarity score between ground-truth and LLM-generated solutions),
SWE-RL enables LLMs to autonomously recover a developer's reasoning processes
and solutions by learning from extensive open-source software evolution data --
the record of a software's entire lifecycle, including its code snapshots, code
changes, and events such as issues and pull requests. Trained on top of Llama
3, our resulting reasoning model, Llama3-SWE-RL-70B, achieves a 41.0% solve
rate on SWE-bench Verified -- a human-verified collection of real-world GitHub
issues. To our knowledge, this is the best performance reported for
medium-sized (<100B) LLMs to date, even comparable to leading proprietary LLMs
like GPT-4o. Surprisingly, despite performing RL solely on software evolution
data, Llama3-SWE-RL has even emerged with generalized reasoning skills. For
example, it shows improved results on five out-of-domain tasks, namely,
function coding, library use, code reasoning, mathematics, and general language
understanding, whereas a supervised-finetuning baseline even leads to
performance degradation on average. Overall, SWE-RL opens up a new direction to
improve the reasoning capabilities of LLMs through reinforcement learning on
massive software engineering data.
67be845a8a5a80542314579f
Yuxiang Wei
admin_assigned
2025-02-26T08:50:44.837Z
632a176259950c1d279d5ea7
https://cdn-avatars.huggingface.co/v1/production/uploads/632a176259950c1d279d5ea7/xsSGhBXalt9RaKzSKY8uk.jpeg
Yuxiang Wei
user
yuxiang630
67be845a8a5a8054231457a0
Olivier Duchenne
67be845a8a5a8054231457a1
Jade Copet
admin_assigned
2025-02-26T08:50:58.290Z
6481e0ac50b759c75d5fdad0
/avatars/49f08d989ca505ae01bce5578a94f6fe.svg
Jade Copet
user
JadeCopet
67be845a8a5a8054231457a2
Quentin Carbonneaux
67be845a8a5a8054231457a3
Lingming Zhang
admin_assigned
2025-02-26T08:51:10.640Z
656f473c14fa8cfccd14559e
/avatars/8f4fef3d835a7a11c2ab66dbf04f3424.svg
Lingming Zhang
user
lingming
67be845a8a5a8054231457a4
Daniel Fried
67be845a8a5a8054231457a5
Gabriel Synnaeve
admin_assigned
2025-02-26T08:51:21.641Z
630eac7931970d1cd4fbacf2
/avatars/b7ccbddfa745db854dc342be1327cd53.svg
Gabriel Synnaeve
user
gsynnaeve
67be845a8a5a8054231457a6
Rishabh Singh
admin_assigned
2025-02-26T08:51:28.321Z
6597e5a6420dcc68501a69e9
/avatars/da48b13e07c367ecd5c891abfd6c3ded.svg
Rishabh Singh
user
RishabhSingh021
67be845a8a5a8054231457a7
Sida I. Wang
SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open
  Software Evolution
The recent DeepSeek-R1 release has demonstrated the immense potential of
reinforcement learning (RL) in enhancing the general reasoning capabilities of
large language models (LLMs). While DeepSeek-R1 and other follow-up work
primarily focus on applying RL to competitive coding and math problems, this
paper introduces SWE-RL, the first approach to scale RL-based LLM reasoning for
real-world software engineering. Leveraging a lightweight rule-based reward
(e.g., the similarity score between ground-truth and LLM-generated solutions),
SWE-RL enables LLMs to autonomously recover a developer's reasoning processes
and solutions by learning from extensive open-source software evolution data --
the record of a software's entire lifecycle, including its code snapshots, code
changes, and events such as issues and pull requests. Trained on top of Llama
3, our resulting reasoning model, Llama3-SWE-RL-70B, achieves a 41.0% solve
rate on SWE-bench Verified -- a human-verified collection of real-world GitHub
issues. To our knowledge, this is the best performance reported for
medium-sized (<100B) LLMs to date, even comparable to leading proprietary LLMs
like GPT-4o. Surprisingly, despite performing RL solely on software evolution
data, Llama3-SWE-RL has even emerged with generalized reasoning skills. For
example, it shows improved results on five out-of-domain tasks, namely,
function coding, library use, code reasoning, mathematics, and general language
understanding, whereas a supervised-finetuning baseline even leads to
performance degradation on average. Overall, SWE-RL opens up a new direction to
improve the reasoning capabilities of LLMs through reinforcement learning on
massive software engineering data.
SWE-RL leverages reinforcement learning to enhance large language models' software engineering reasoning, achieving state-of-the-art performance and generalized reasoning skills by learning from extensive open-source software evolution data.
reinforcement learning large language models RL-based LLM reasoning lightweight rule-based reward open-source software evolution data SWE-bench Verified solve rate Llama 3 function coding library use code reasoning mathematics general language understanding