Training Long-Context, Multi-Turn Software Engineering Agents with
  Reinforcement Learning
Research on applications of Reinforcement Learning (RL) to Large Language
Models (LLMs) has mostly been focused on single-turn problems, such as
mathematical reasoning or single-shot code generation. While these problems can
be viewed as token-level multi-turn MDPs, this view corresponds to a degenerate
case of multi-turn interaction where the environment provides no feedback. This
contrasts with many real-world domains, such as software engineering (SWE),
which require rich multi-turn interactions with a stateful environment that
responds to each action with a non-trivial observation.
  To bridge this gap, we demonstrate the successful application of RL to this
general regime. Using a modified Decoupled Advantage Policy Optimization (DAPO)
algorithm, we train an agent based on Qwen2.5-72B-Instruct to solve real-world
software engineering tasks. Our approach increases the agent's success rate on
the SWE-bench Verified benchmark from a 20% rejection fine-tuned baseline to
39%, without relying on any teacher models. On SWE-rebench, our agent matches
or outperforms leading open-weight models such as DeepSeek-V3-0324 and
Qwen3-235B-A22B using an identical scaffolding, offering a viable path toward
building more capable autonomous agents for complex real-world problems based
on open models.
68946454741a16f544fbd048
Alexander Golubev
claimed_verified
2025-08-08T16:21:25.178Z
644e9ffcd6001776ed77d874
/avatars/b93e02caf929679b7e9bc589eed0b689.svg
Alexander
user
djalexj
68946454741a16f544fbd049
Maria Trofimova
68946454741a16f544fbd04a
Sergei Polezhaev
68946454741a16f544fbd04b
Ibragim Badertdinov
claimed_verified
2025-08-08T16:21:29.173Z
654e5e094319c75e3e1b6cbc
/avatars/a8889036fa38f80f2d45aea8d1471395.svg
Ibragim
user
ibragim-bad
68946454741a16f544fbd04c
Maksim Nekrashevich
68946454741a16f544fbd04d
Anton Shevtsov
claimed_verified
2025-08-13T07:27:58.754Z
66214f06d55e5f677e0da9f2
/avatars/dc3b8de4a72b0d10f6bb90594488a9ec.svg
Anton
user
mrshevan
68946454741a16f544fbd04e
Simon Karasik
claimed_verified
2025-08-08T16:21:32.350Z
65e48cb3a4e46e644ec1277d
/avatars/dd2bf04a6f81bf0a0892080af5d485b2.svg
Simon Karasik
user
sbkarasik
68946454741a16f544fbd04f
Sergey Abramov
68946454741a16f544fbd050
Andrei Andriushchenko
68946454741a16f544fbd051
Filipp Fisin
68946454741a16f544fbd052
Sergei Skvortsov
68946454741a16f544fbd053
Boris Yangel
Training Long-Context, Multi-Turn Software Engineering Agents with
  Reinforcement Learning
Research on applications of Reinforcement Learning (RL) to Large Language
Models (LLMs) has mostly been focused on single-turn problems, such as
mathematical reasoning or single-shot code generation. While these problems can
be viewed as token-level multi-turn MDPs, this view corresponds to a degenerate
case of multi-turn interaction where the environment provides no feedback. This
contrasts with many real-world domains, such as software engineering (SWE),
which require rich multi-turn interactions with a stateful environment that
responds to each action with a non-trivial observation.
  To bridge this gap, we demonstrate the successful application of RL to this
general regime. Using a modified Decoupled Advantage Policy Optimization (DAPO)
algorithm, we train an agent based on Qwen2.5-72B-Instruct to solve real-world
software engineering tasks. Our approach increases the agent's success rate on
the SWE-bench Verified benchmark from a 20% rejection fine-tuned baseline to
39%, without relying on any teacher models. On SWE-rebench, our agent matches
or outperforms leading open-weight models such as DeepSeek-V3-0324 and
Qwen3-235B-A22B using an identical scaffolding, offering a viable path toward
building more capable autonomous agents for complex real-world problems based
on open models.