Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in
  LLMs
Software engineering (SWE) has recently emerged as a crucial testbed for
next-generation LLM agents, demanding inherent capabilities in two critical
dimensions: sustained iterative problem-solving (e.g., >50 interaction rounds)
and long-context dependency resolution (e.g., >32k tokens). However, the data
curation process in SWE remains notoriously time-consuming, as it heavily
relies on manual annotation for code file filtering and the setup of dedicated
runtime environments to execute and validate unit tests. Consequently, most
existing datasets are limited to only a few thousand GitHub-sourced instances.
To this end, we propose an incremental, automated data-curation pipeline that
systematically scales both the volume and diversity of SWE datasets. Our
dataset comprises 10,169 real-world Python task instances from 2,531 distinct
GitHub repositories, each accompanied by a task specified in natural language
and a dedicated runtime-environment image for automated unit-test validation.
We have carefully curated over 8,000 successfully runtime-validated training
trajectories from our proposed SWE dataset. When fine-tuning the Skywork-SWE
model on these trajectories, we uncover a striking data scaling phenomenon: the
trained model's performance for software engineering capabilities in LLMs
continues to improve as the data size increases, showing no signs of
saturation. Notably, our Skywork-SWE model achieves 38.0% pass@1 accuracy on
the SWE-bench Verified benchmark without using verifiers or multiple rollouts,
establishing a new state-of-the-art (SOTA) among the Qwen2.5-Coder-32B-based
LLMs built on the OpenHands agent framework. Furthermore, with the
incorporation of test-time scaling techniques, the performance further improves
to 47.0% accuracy, surpassing the previous SOTA results for sub-32B parameter
models. We release the Skywork-SWE-32B model checkpoint to accelerate future
research.
685b6640d2ee4fac76521e42
Liang Zeng
claimed_verified
2025-06-25T08:08:30.529Z
6621efe1a6eec3ad03e38759
/avatars/c35acce69f244ec0833dffd53eedf6a3.svg
Liang Zeng
user
zengliangcs
685b6640d2ee4fac76521e43
Yongcong Li
claimed_verified
2025-06-25T08:08:28.275Z
612cfc6e1f69b222aacf831b
/avatars/b6c7d15ebc7b5dd4b56620bfab324c77.svg
lycfight
user
lycfight
685b6640d2ee4fac76521e44
Yuzhen Xiao
admin_assigned
2025-06-27T14:34:25.408Z
658ceb595a8f8a309ea417a1
/avatars/43bfa8c919aa802f2611439ebb7430b8.svg
Ricky Shaw
user
RickyShaw999
685b6640d2ee4fac76521e45
Changshi Li
claimed_verified
2025-06-27T08:57:48.619Z
6665dd2b3a64c70529f7542c
/avatars/9b12a02c429ed20d136b9def8d3f9221.svg
Changshi Li
user
ChangshiLi
685b6640d2ee4fac76521e46
Chris Yuhao Liu
claimed_verified
2025-06-25T08:08:22.815Z
658229ef5f6d83438257fce5
/avatars/b4417de9a338e95dc69cc547a46348e8.svg
Chris (Yuhao) Liu
user
chrisliu298
685b6640d2ee4fac76521e47
Rui Yan
685b6640d2ee4fac76521e48
Tianwen Wei
685b6640d2ee4fac76521e49
Jujie He
685b6640d2ee4fac76521e4a
Xuchen Song
685b6640d2ee4fac76521e4b
Yang Liu
685b6640d2ee4fac76521e4c
Yahui Zhou
Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in
  LLMs
Software engineering (SWE) has recently emerged as a crucial testbed for
next-generation LLM agents, demanding inherent capabilities in two critical
dimensions: sustained iterative problem-solving (e.g., >50 interaction rounds)
and long-context dependency resolution (e.g., >32k tokens). However, the data
curation process in SWE remains notoriously time-consuming, as it heavily
relies on manual annotation for code file filtering and the setup of dedicated
runtime environments to execute and validate unit tests. Consequently, most
existing datasets are limited to only a few thousand GitHub-sourced instances.
To this end, we propose an incremental, automated data-curation pipeline that
systematically scales both the volume and diversity of SWE datasets. Our
dataset comprises 10,169 real-world Python task instances from 2,531 distinct
GitHub repositories, each accompanied by a task specified in natural language
and a dedicated runtime-environment image for automated unit-test validation.
We have carefully curated over 8,000 successfully runtime-validated training
trajectories from our proposed SWE dataset. When fine-tuning the Skywork-SWE
model on these trajectories, we uncover a striking data scaling phenomenon: the
trained model's performance for software engineering capabilities in LLMs
continues to improve as the data size increases, showing no signs of
saturation. Notably, our Skywork-SWE model achieves 38.0% pass@1 accuracy on
the SWE-bench Verified benchmark without using verifiers or multiple rollouts,
establishing a new state-of-the-art (SOTA) among the Qwen2.5-Coder-32B-based
LLMs built on the OpenHands agent framework. Furthermore, with the
incorporation of test-time scaling techniques, the performance further improves
to 47.0% accuracy, surpassing the previous SOTA results for sub-32B parameter
models. We release the Skywork-SWE-32B model checkpoint to accelerate future
research.
An automated data-curation pipeline for software engineering improves large language model performance on SWE tasks, achieving state-of-the-art results with and without test-time scaling techniques.
LLM agents iterative problem-solving long-context dependency resolution code file filtering unit tests runtime environments data-curation pipeline software engineering capabilities Skywork-SWE model SWE-bench Verified pass@1 accuracy OpenHands agent framework test-time scaling techniques parameter-efficient fine-tuning