Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers
Recent advances in Large Language Models (LLMs) have shown that their
reasoning capabilities can be significantly improved through Reinforcement
Learning with Verifiable Reward (RLVR), particularly in domains like
mathematics and programming, where ground-truth correctness can be
automatically evaluated. However, extending this success to other
reasoning-intensive domains remains challenging due to the scarcity of
high-quality, verifiable datasets and the high cost of human supervision. In
this work, we introduce the Loong Project: an open-source framework for
scalable synthetic data generation and verification across a diverse range of
reasoning-intensive domains. The framework consists of two key components: (1)
LoongBench, a curated seed dataset containing 8,729 human-vetted examples
across 12 domains (e.g., Advanced Mathematics, Chemistry, Logic), each paired
with executable code and rich metadata; and (2) LoongEnv, a modular synthetic
data generation environment that supports multiple prompting strategies to
produce new question-answer-code triples. Together, these components form an
agent-environment loop that enables reinforcement learning, where an LLM-based
agent is rewarded for generating Chain-of-Thought (CoT) solutions that align
with code-executed answers. Empirically, we benchmark LoongBench on a broad
suite of both open-source and proprietary LLMs to evaluate domain coverage and
reveal performance bottlenecks. In addition, we conduct a comprehensive
analysis of synthetic data generated by LoongEnv, examining correctness,
difficulty, and diversity. Code and documentation are available at
https://github.com/camel-ai/loong.
68b913eed43cadaf7a688b6c
Xingyue Huang
claimed_verified
2025-09-04T08:43:53.847Z
62cb06206193ba3ced7afa05
/avatars/3265c0523642b2463b7c6c7a4181681c.svg
Xingyue Huang
user
hxyscott
68b913eed43cadaf7a688b6d
Rishabh
68b913eed43cadaf7a688b6e
Gregor Franke
68b913eed43cadaf7a688b6f
Ziyi Yang
68b913eed43cadaf7a688b70
Jiamu Bai
68b913eed43cadaf7a688b71
Weijie Bai
68b913eed43cadaf7a688b72
Jinhe Bi
68b913eed43cadaf7a688b73
Zifeng Ding
68b913eed43cadaf7a688b74
Yiqun Duan
68b913eed43cadaf7a688b75
Chengyu Fan
68b913eed43cadaf7a688b76
Wendong Fan
68b913eed43cadaf7a688b77
Xin Gao
68b913eed43cadaf7a688b78
Ruohao Guo
68b913eed43cadaf7a688b79
Yuan He
68b913eed43cadaf7a688b7a
Zhuangzhuang He
admin_assigned
2025-09-22T19:40:51.928Z
6621e88fa11ce46061d25a16
/avatars/4a9a965a2d0f33e2855d2909a3e162bc.svg
Zhuangzhuang He
user
bruno888
68b913eed43cadaf7a688b7b
Xianglong Hu
68b913eed43cadaf7a688b7c
Neil Johnson
68b913eed43cadaf7a688b7d
Bowen Li
68b913eed43cadaf7a688b7e
Fangru Lin
68b913eed43cadaf7a688b7f
Siyu Lin
68b913eed43cadaf7a688b80
Tong Liu
68b913eed43cadaf7a688b81
Yunpu Ma
68b913eed43cadaf7a688b82
Hao Shen
68b913eed43cadaf7a688b83
Hao Sun
68b913eed43cadaf7a688b84
Beibei Wang
68b913eed43cadaf7a688b85
Fangyijie Wang
claimed_verified
2025-09-06T10:57:21.473Z
6401f9193e3d0f2745ae7760
/avatars/2f978944e290052a72c9d7e1e14a68c2.svg
FANGYIJIE WANG
user
fangyijie
68b913eed43cadaf7a688b86
Hao Wang
68b913eed43cadaf7a688b87
Haoran Wang
68b913eed43cadaf7a688b88
Yang Wang
68b913eed43cadaf7a688b89
Yifeng Wang
68b913eed43cadaf7a688b8a
Zhaowei Wang
68b913eed43cadaf7a688b8b
Ziyang Wang
68b913eed43cadaf7a688b8c
Yifan Wu
68b913eed43cadaf7a688b8d
Zikai Xiao
68b913eed43cadaf7a688b8e
Chengxing Xie
68b913eed43cadaf7a688b8f
Fan Yang
68b913eed43cadaf7a688b90
Junxiao Yang
claimed_verified
2025-09-04T12:51:29.776Z
65d859a3661492b25c46a117
/avatars/06a547a00b472f512a25eef4ddd047bf.svg
Junxiao Yang
user
yangjunxiao2021
68b913eed43cadaf7a688b91
Qianshuo Ye
68b913eed43cadaf7a688b92
Ziyu Ye
68b913eed43cadaf7a688b93
Guangtao Zeng
68b913eed43cadaf7a688b94
Yuwen Ebony Zhang
68b913eed43cadaf7a688b95
Zeyu Zhang
68b913eed43cadaf7a688b96
Zihao Zhu
claimed_verified
2025-09-04T12:51:31.659Z
64030d9956038547951c7d55
https://cdn-avatars.huggingface.co/v1/production/uploads/64030d9956038547951c7d55/gRCzhy1yd7cOV6GN-KGHJ.png
Zihao Zhu
user
ZihaoZhu
68b913eed43cadaf7a688b97
Bernard Ghanem
68b913eed43cadaf7a688b98
Philip Torr
68b913eed43cadaf7a688b99
Guohao Li
claimed_verified
2025-09-06T10:57:23.571Z
6338790e76421c054310c96b
/avatars/112e3d88d155bc998a89fef6f33af64d.svg
Guohao Li
user
lightaime
Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers
Recent advances in Large Language Models (LLMs) have shown that their
reasoning capabilities can be significantly improved through Reinforcement
Learning with Verifiable Reward (RLVR), particularly in domains like
mathematics and programming, where ground-truth correctness can be
automatically evaluated. However, extending this success to other
reasoning-intensive domains remains challenging due to the scarcity of
high-quality, verifiable datasets and the high cost of human supervision. In
this work, we introduce the Loong Project: an open-source framework for
scalable synthetic data generation and verification across a diverse range of
reasoning-intensive domains. The framework consists of two key components: (1)
LoongBench, a curated seed dataset containing 8,729 human-vetted examples
across 12 domains (e.g., Advanced Mathematics, Chemistry, Logic), each paired
with executable code and rich metadata; and (2) LoongEnv, a modular synthetic
data generation environment that supports multiple prompting strategies to
produce new question-answer-code triples. Together, these components form an
agent-environment loop that enables reinforcement learning, where an LLM-based
agent is rewarded for generating Chain-of-Thought (CoT) solutions that align
with code-executed answers. Empirically, we benchmark LoongBench on a broad
suite of both open-source and proprietary LLMs to evaluate domain coverage and
reveal performance bottlenecks. In addition, we conduct a comprehensive
analysis of synthetic data generated by LoongEnv, examining correctness,
difficulty, and diversity. Code and documentation are available at
https://github.com/camel-ai/loong.
The Loong Project introduces a framework for generating and verifying synthetic data to improve reasoning capabilities in Large Language Models through Reinforcement Learning with Verifiable Reward.
Large Language Models Reinforcement Learning with Verifiable Reward synthetic data generation verification LoongBench LoongEnv Chain-of-Thought domain coverage performance bottlenecks correctness difficulty diversity