Kimi k1.5: Scaling Reinforcement Learning with LLMs
Language model pretraining with next token prediction has proved effective
for scaling compute but is limited to the amount of available training data.
Scaling reinforcement learning (RL) unlocks a new axis for the continued
improvement of artificial intelligence, with the promise that large language
models (LLMs) can scale their training data by learning to explore with
rewards. However, prior published work has not produced competitive results. In
light of this, we report on the training practice of Kimi k1.5, our latest
multi-modal LLM trained with RL, including its RL training techniques,
multi-modal data recipes, and infrastructure optimization. Long context scaling
and improved policy optimization methods are key ingredients of our approach,
which establishes a simplistic, effective RL framework without relying on more
complex techniques such as Monte Carlo tree search, value functions, and
process reward models. Notably, our system achieves state-of-the-art reasoning
performance across multiple benchmarks and modalities -- e.g., 77.5 on AIME,
96.2 on MATH 500, 94-th percentile on Codeforces, 74.9 on MathVista -- matching
OpenAI's o1. Moreover, we present effective long2short methods that use
long-CoT techniques to improve short-CoT models, yielding state-of-the-art
short-CoT reasoning results -- e.g., 60.8 on AIME, 94.6 on MATH500, 47.3 on
LiveCodeBench -- outperforming existing short-CoT models such as GPT-4o and
Claude Sonnet 3.5 by a large margin (up to +550%).
6791b6029e215712a7cf700a
Kimi Team
6791b6029e215712a7cf700b
Angang Du
claimed_verified
2025-05-08T09:00:06.815Z
617798384ce8f8cb2c2c74a3
https://cdn-avatars.huggingface.co/v1/production/uploads/617798384ce8f8cb2c2c74a3/cWSTlcZrpyEf4shuVEcdW.png
Ang
user
duang
6791b6029e215712a7cf700c
Bofei Gao
admin_assigned
2025-01-23T09:41:32.895Z
65ae21adabf6d1ccb795e9a4
/avatars/b5dced62c6a3564095a8fa0959bc06cb.svg
Bofei Gao
user
KbsdJames
6791b6029e215712a7cf700d
Bowei Xing
admin_assigned
2025-01-23T09:41:55.675Z
67503f270dfe827c4068a408
/avatars/4591c8229c7815bfd6dc4b98aea85ca8.svg
Bowei Xing
user
xingbowei
6791b6029e215712a7cf700e
Changjiu Jiang
6791b6029e215712a7cf700f
Cheng Chen
6791b6029e215712a7cf7010
Cheng Li
6791b6029e215712a7cf7011
Chenjun Xiao
admin_assigned
2025-01-23T09:42:24.770Z
66c9aa8fa40c9235cb446365
/avatars/616719ed848e1f8437fb39e6184feb52.svg
Chenjun Xiao
user
shelowize
6791b6029e215712a7cf7012
Chenzhuang Du
admin_assigned
2025-01-23T09:42:36.347Z
64c21fb42426d683e56b42bf
/avatars/60359fe204e32af831d701d2975c4599.svg
Du
user
DuChenZhuang
6791b6029e215712a7cf7013
Chonghua Liao
admin_assigned
2025-01-23T09:43:07.678Z
667fdaee20ee9ac417c7708c
/avatars/69dfba6ff392643af1dcfe8af0a42ae9.svg
Chonghua Liao
user
ChonghuaLiao
6791b6029e215712a7cf7014
Chuning Tang
6791b6029e215712a7cf7015
Congcong Wang
admin_assigned
2025-01-23T09:43:24.138Z
5eefd87c5e979253a010eee5
https://cdn-avatars.huggingface.co/v1/production/uploads/1603575136094-5eefd87c5e979253a010eee5.jpeg
Congcong Wang
user
congcongwang
6791b6029e215712a7cf7016
Dehao Zhang
admin_assigned
2025-01-23T09:44:01.384Z
634d0eeffc4873ed0d00406a
https://cdn-avatars.huggingface.co/v1/production/uploads/1665994473058-noauth.jpeg
ZhangDehao
user
ispoon
6791b6029e215712a7cf7017
Enming Yuan
admin_assigned
2025-01-23T09:44:09.134Z
6331606f18711776b4655e67
/avatars/1479c2ca743b9f92d845b0ed23fcd07b.svg
Enming Yuan
user
EnmingYuan
6791b6029e215712a7cf7018
Enzhe Lu
6791b6029e215712a7cf7019
Fengxiang Tang
6791b6029e215712a7cf701a
Flood Sung
admin_assigned
2025-01-23T09:44:34.078Z
6343d01a08c017b2c042305d
https://cdn-avatars.huggingface.co/v1/production/uploads/6343d01a08c017b2c042305d/cmJrYkGs9RjDAKfMYCqdW.jpeg
Flood Sung
user
floodsung
6791b6029e215712a7cf701b
Guangda Wei
6791b6029e215712a7cf701c
Guokun Lai
admin_assigned
2025-01-23T09:44:52.466Z
63b4c71758f367a212c4f9ef
/avatars/d61736e0ae8b333a7c24eb411378698c.svg
Lai
user
Guokun
6791b6029e215712a7cf701d
Haiqing Guo
6791b6029e215712a7cf701e
Han Zhu
6791b6029e215712a7cf701f
Hao Ding
6791b6029e215712a7cf7020
Hao Hu
6791b6029e215712a7cf7021
Hao Yang
claimed_verified
2025-01-23T15:05:07.301Z
64ec364e7e2ec711a7601cde
/avatars/6ba47d496586de65df183f056d35982b.svg
Hao Yang
user
hayayanghao
6791b6029e215712a7cf7022
Hao Zhang
6791b6029e215712a7cf7023
Haotian Yao
6791b6029e215712a7cf7024
Haotian Zhao
6791b6029e215712a7cf7025
Haoyu Lu
claimed_verified
2025-06-11T08:40:02.004Z
64c206a3fd4d5966b453ed85
https://cdn-avatars.huggingface.co/v1/production/uploads/64c206a3fd4d5966b453ed85/NemBrHcAJFm2ws_VQG8ia.jpeg
Haoyu Lu
user
Nealeon
6791b6029e215712a7cf7026
Haoze Li
6791b6029e215712a7cf7027
Haozhen Yu
6791b6029e215712a7cf7028
Hongcheng Gao
admin_assigned
2025-01-23T09:47:40.340Z
62728f4f6253fe2068da1021
https://cdn-avatars.huggingface.co/v1/production/uploads/62728f4f6253fe2068da1021/KZ65X0EH98AF3zXemPiap.jpeg
Hongcheng Gao
user
HongchengGao
6791b6029e215712a7cf7029
Huabin Zheng
admin_assigned
2025-01-23T09:47:47.850Z
61860e1258cb1f8c362f9441
/avatars/8dbc8209ad0d918453c1ffacc8f61e7f.svg
Huabin Zheng
user
zhenghuabin
6791b6029e215712a7cf702a
Huan Yuan
6791b6029e215712a7cf702b
Jia Chen
6791b6029e215712a7cf702c
Jianhang Guo
6791b6029e215712a7cf702d
Jianlin Su
6791b6029e215712a7cf702e
Jianzhou Wang
admin_assigned
2025-01-23T09:48:00.137Z
63be6bf6da08ed0544f1eb7a
/avatars/19b5be6d3296da402d8822e51d6376e2.svg
jianzhouWang
user
jianzhouWang
6791b6029e215712a7cf702f
Jie Zhao
6791b6029e215712a7cf7030
Jin Zhang
6791b6029e215712a7cf7031
Jingyuan Liu
6791b6029e215712a7cf7032
Junjie Yan
6791b6029e215712a7cf7033
Junyan Wu
6791b6029e215712a7cf7034
Lidong Shi
6791b6029e215712a7cf7035
Ling Ye
6791b6029e215712a7cf7036
Longhui Yu
claimed_verified
2025-01-24T10:01:45.720Z
64b753306c169983c982f609
https://cdn-avatars.huggingface.co/v1/production/uploads/64b753306c169983c982f609/oOC_N8cI1Go6avUMvIPVI.jpeg
Longhui Yu
user
Longhui98
6791b6029e215712a7cf7037
Mengnan Dong
6791b6029e215712a7cf7038
Neo Zhang
6791b6029e215712a7cf7039
Ningchen Ma
6791b6029e215712a7cf703a
Qiwei Pan
6791b6029e215712a7cf703b
Qucheng Gong
6791b6029e215712a7cf703c
Shaowei Liu
6791b6029e215712a7cf703d
Shengling Ma
6791b6029e215712a7cf703e
Shupeng Wei
6791b6029e215712a7cf703f
Sihan Cao
6791b6029e215712a7cf7040
Siying Huang
6791b6029e215712a7cf7041
Tao Jiang
6791b6029e215712a7cf7042
Weihao Gao
6791b6029e215712a7cf7043
Weimin Xiong
claimed_verified
2025-02-18T09:35:12.836Z
6225a9983207dfc568407204
/avatars/c970db6232d84ae8c0fa5f11d561d67c.svg
xwm
user
xwm
6791b6029e215712a7cf7044
Weiran He
claimed_verified
2025-04-15T08:04:02.280Z
63a41c1f412fd71fb7ec4a20
/avatars/d6837b0dbbf3507896e98754d3f8a468.svg
Wayne Ho
user
hewr2010
6791b6029e215712a7cf7045
Weixiao Huang
6791b6029e215712a7cf7046
Wenhao Wu
6791b6029e215712a7cf7047
Wenyang He
6791b6029e215712a7cf7048
Xianghui Wei
6791b6029e215712a7cf7049
Xianqing Jia
6791b6029e215712a7cf704a
Xingzhe Wu
6791b6029e215712a7cf704b
Xinran Xu
6791b6029e215712a7cf704c
Xinxing Zu
6791b6029e215712a7cf704d
Xinyu Zhou
6791b6029e215712a7cf704e
Xuehai Pan
6791b6029e215712a7cf704f
Y. Charles
6791b6029e215712a7cf7050
Yang Li
6791b6029e215712a7cf7051
Yangyang Hu
6791b6029e215712a7cf7052
Yangyang Liu
6791b6029e215712a7cf7053
Yanru Chen
6791b6029e215712a7cf7054
Yejie Wang
6791b6029e215712a7cf7055
Yibo Liu
6791b6029e215712a7cf7056
Yidao Qin
6791b6029e215712a7cf7057
Yifeng Liu
claimed_verified
2025-02-19T09:05:48.369Z
653d276681f52ceb4d12bd85
/avatars/56601a25e5f883a8f6dc15f6fd9dcc57.svg
Yifeng Liu
user
Lewis-Lau
6791b6029e215712a7cf7058
Ying Yang
6791b6029e215712a7cf7059
Yiping Bao
6791b6029e215712a7cf705a
Yulun Du
admin_assigned
2025-01-23T09:47:18.988Z
6340f31fb78ed99eab04ce33
/avatars/2e7fcbf0233bdc0bc9a3f4603fd8bf90.svg
Du
user
Yulun
6791b6029e215712a7cf705b
Yuxin Wu
6791b6029e215712a7cf705c
Yuzhi Wang
admin_assigned
2025-01-23T09:46:54.213Z
67127a470a82509269d738ae
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/M9qLmI3P6dT2FIwEPFJq0.png
yuzhi wang
user
vin-tage
6791b6029e215712a7cf705d
Zaida Zhou
admin_assigned
2025-01-23T09:46:46.940Z
64409d69518271b0d1c033a6
/avatars/c79eb36c4ad96286afda834e260a1c09.svg
zhouzaida
user
zhouzaida
6791b6029e215712a7cf705e
Zhaoji Wang
6791b6029e215712a7cf705f
Zhaowei Li
6791b6029e215712a7cf7060
Zhen Zhu
6791b6029e215712a7cf7061
Zheng Zhang
6791b6029e215712a7cf7062
Zhexu Wang
admin_assigned
2025-01-23T09:46:16.399Z
645b8bfd78730bcc103f9757
/avatars/1f9b7ec00744708ab609b1c4db2c2fcc.svg
Wang Zhexuan
user
longmao14
6791b6029e215712a7cf7063
Zhilin Yang
admin_assigned
2025-01-23T09:46:04.605Z
64bf74154d2052b1aa5ca6d9
/avatars/7aa6f2952cdbc20cfa758fdd905f06a6.svg
ZHILIN YANG
user
bruceyannnn
6791b6029e215712a7cf7064
Zhiqi Huang
6791b6029e215712a7cf7065
Zihao Huang
6791b6029e215712a7cf7066
Ziyao Xu
admin_assigned
2025-01-23T09:45:24.566Z
64d381c3527b22502ec81eb7
/avatars/55145a716acf120a32a302830c1e8178.svg
Ziyao Xu
user
xzyxzy
6791b6029e215712a7cf7067
Zonghan Yang
admin_assigned
2025-01-23T09:45:16.573Z
635795f24fba057662cdf2e7
/avatars/07642c7cfd9b57a0f8382268b009cf6f.svg
Zonghan Yang
user
minicheshire
Kimi k1.5: Scaling Reinforcement Learning with LLMs
Language model pretraining with next token prediction has proved effective
for scaling compute but is limited to the amount of available training data.
Scaling reinforcement learning (RL) unlocks a new axis for the continued
improvement of artificial intelligence, with the promise that large language
models (LLMs) can scale their training data by learning to explore with
rewards. However, prior published work has not produced competitive results. In
light of this, we report on the training practice of Kimi k1.5, our latest
multi-modal LLM trained with RL, including its RL training techniques,
multi-modal data recipes, and infrastructure optimization. Long context scaling
and improved policy optimization methods are key ingredients of our approach,
which establishes a simplistic, effective RL framework without relying on more
complex techniques such as Monte Carlo tree search, value functions, and
process reward models. Notably, our system achieves state-of-the-art reasoning
performance across multiple benchmarks and modalities -- e.g., 77.5 on AIME,
96.2 on MATH 500, 94-th percentile on Codeforces, 74.9 on MathVista -- matching
OpenAI's o1. Moreover, we present effective long2short methods that use
long-CoT techniques to improve short-CoT models, yielding state-of-the-art
short-CoT reasoning results -- e.g., 60.8 on AIME, 94.6 on MATH500, 47.3 on
LiveCodeBench -- outperforming existing short-CoT models such as GPT-4o and
Claude Sonnet 3.5 by a large margin (up to +550%).
A multi-modal LLM trained with reinforcement learning achieves state-of-the-art reasoning performance across various benchmarks by utilizing long context scaling and effective policy optimization methods.
next token prediction reinforcement learning large language models multi-modal LLM RL training techniques multi-modal data recipes infrastructure optimization long context scaling policy optimization Monte Carlo tree search value functions process reward models AIME MATH 500 Codeforces MathVista long-CoT techniques short-CoT models short-CoT reasoning