Qwen2.5 Technical Report
In this report, we introduce Qwen2.5, a comprehensive series of large
language models (LLMs) designed to meet diverse needs. Compared to previous
iterations, Qwen 2.5 has been significantly improved during both the
pre-training and post-training stages. In terms of pre-training, we have scaled
the high-quality pre-training datasets from the previous 7 trillion tokens to
18 trillion tokens. This provides a strong foundation for common sense, expert
knowledge, and reasoning capabilities. In terms of post-training, we implement
intricate supervised finetuning with over 1 million samples, as well as
multistage reinforcement learning. Post-training techniques enhance human
preference, and notably improve long text generation, structural data analysis,
and instruction following. To handle diverse and varied use cases effectively,
we present Qwen2.5 LLM series in rich sizes. Open-weight offerings include base
and instruction-tuned models, with quantized versions available. In addition,
for hosted solutions, the proprietary models currently include two
mixture-of-experts (MoE) variants: Qwen2.5-Turbo and Qwen2.5-Plus, both
available from Alibaba Cloud Model Studio. Qwen2.5 has demonstrated top-tier
performance on a wide range of benchmarks evaluating language understanding,
reasoning, mathematics, coding, human preference alignment, etc. Specifically,
the open-weight flagship Qwen2.5-72B-Instruct outperforms a number of open and
proprietary models and demonstrates competitive performance to the
state-of-the-art open-weight model, Llama-3-405B-Instruct, which is around 5
times larger. Qwen2.5-Turbo and Qwen2.5-Plus offer superior cost-effectiveness
while performing competitively against GPT-4o-mini and GPT-4o respectively.
Additionally, as the foundation, Qwen2.5 models have been instrumental in
training specialized models such as Qwen2.5-Math, Qwen2.5-Coder, QwQ, and
multimodal models.
6764dda159e2ef96ec013c07
Qwen
6764dda159e2ef96ec013c09
An Yang
claimed_verified
2025-05-19T06:43:12.521Z
62088594a5943c8a8fc94560
https://cdn-avatars.huggingface.co/v1/production/uploads/1644733028938-62088594a5943c8a8fc94560.png
An Yang
user
yangapku
6764dda159e2ef96ec013c0a
Baosong Yang
admin_assigned
2024-12-20T08:42:54.655Z
64b0a77df12b47366663884c
/avatars/a212ea862abb5966060e439dd0e7656f.svg
Baosong Yang
user
Baosong
6764dda159e2ef96ec013c0b
Beichen Zhang
6764dda159e2ef96ec013c0c
Binyuan Hui
claimed_verified
2024-12-20T08:32:55.260Z
61e4c4ca1ab24785ac11ba69
https://cdn-avatars.huggingface.co/v1/production/uploads/61e4c4ca1ab24785ac11ba69/1Q1zhhyGSJ9RJG9MzwxVv.jpeg
Binyuan Hui
user
huybery
6764dda159e2ef96ec013c0d
Bo Zheng
admin_assigned
2024-12-20T09:15:45.070Z
62c695ad5aae1c624ca992e2
/avatars/20d10fb3338e4bd4dc59e88a18cb2617.svg
Bo Zheng
user
bzheng
6764dda159e2ef96ec013c0e
Bowen Yu
admin_assigned
2024-12-20T08:43:56.820Z
6583ab7983a9e1460c67d876
/avatars/74400bc448c3f07e23a4cd53d68a6af7.svg
bowen
user
bowenYu
6764dda159e2ef96ec013c0f
Chengyuan Li
6764dda159e2ef96ec013c10
Dayiheng Liu
claimed_verified
2024-12-20T08:32:51.398Z
6434d4989bd5a84b5dd0b0f5
https://cdn-avatars.huggingface.co/v1/production/uploads/6434d4989bd5a84b5dd0b0f5/0Elf9qbfG9Hkgypm9pTGm.jpeg
Dayiheng Liu
user
Losin94
6764dda159e2ef96ec013c11
Fei Huang
6764dda159e2ef96ec013c12
Haoran Wei
admin_assigned
2024-12-20T08:44:49.831Z
6436618aeef1f55654a9f458
https://cdn-avatars.huggingface.co/v1/production/uploads/6436618aeef1f55654a9f458/OvxGtuDg2GAFG9As-2hzW.jpeg
Haoran Wei
user
HaoranWei
6764dda159e2ef96ec013c13
Huan Lin
6764dda159e2ef96ec013c14
Jian Yang
6764dda159e2ef96ec013c15
Jianhong Tu
admin_assigned
2024-12-20T08:45:05.864Z
654bead777401b47e6424f88
/avatars/7bcbdbb051c93b004f0dc3ad36c4a0ce.svg
Jianhong Tu
user
ToviTu
6764dda159e2ef96ec013c16
Jianwei Zhang
6764dda159e2ef96ec013c17
Jianxin Yang
6764dda159e2ef96ec013c18
Jiaxi Yang
6764dda159e2ef96ec013c19
Jingren Zhou
admin_assigned
2024-12-20T08:45:52.281Z
602f88f5e8149a962412a667
/avatars/b78f0e583df8e5d5e3365934fe5f4900.svg
Zhou
user
Jingren
6764dda159e2ef96ec013c1a
Junyang Lin
admin_assigned
2024-12-20T08:45:59.082Z
620760a26e3b7210c2ff1943
https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/VC-rKqimF6yxGESNVlPoR.jpeg
Junyang Lin
user
JustinLin610
6764dda159e2ef96ec013c1b
Kai Dang
admin_assigned
2024-12-20T15:42:29.190Z
6712930f0fac3235c56edf5b
/avatars/cafe7cb56ce7c3b2572f5f2d0b89357a.svg
kai dang
user
1vk5i
6764dda159e2ef96ec013c1c
Keming Lu
admin_assigned
2024-12-20T08:46:07.173Z
6453fa96ed6d7fede94408e0
/avatars/e8c9025ef24cec958c87a1008bb54fd7.svg
Keming Lu
user
keminglu
6764dda159e2ef96ec013c1d
Keqin Bao
admin_assigned
2024-12-20T08:42:00.963Z
6154629f6181394cc00cb98b
/avatars/de2d5cb6fee9c03a99289d03272eddeb.svg
Keqin Bao
user
morningStar
6764dda159e2ef96ec013c1e
Kexin Yang
admin_assigned
2024-12-20T08:46:29.866Z
65b0b3957e5d5a4ecc750de0
/avatars/e0d79d3265ca4ad5c5411feb01043fb4.svg
Kexin Yang
user
dawn0929
6764dda159e2ef96ec013c1f
Le Yu
claimed_verified
2025-05-18T08:48:05.785Z
6455056fe4952d1c6cb45e49
/avatars/bbdcd4405497f11ff3774352277d1ec7.svg
Le Yu
user
vanillaOVO
6764dda159e2ef96ec013c20
Mei Li
6764dda159e2ef96ec013c21
Mingfeng Xue
admin_assigned
2024-12-20T08:46:45.959Z
5f8946925d083370c711f296
/avatars/14246aae3b1f8b7ad050f8ff2c8b260e.svg
Mingfeng Xue
user
mingfengxue
6764dda159e2ef96ec013c22
Pei Zhang
admin_assigned
2024-12-20T08:47:05.519Z
634872721df68daaa0125e08
/avatars/c091b70448a4e1d43fce3a31a7beddf3.svg
Pei Zhang
user
peizhang
6764dda159e2ef96ec013c23
Qin Zhu
6764dda159e2ef96ec013c24
Rui Men
6764dda159e2ef96ec013c25
Runji Lin
claimed_verified
2024-12-20T08:32:53.455Z
649a52e5de0fb7f3f499e583
/avatars/25f6106fa168ae57ad5cd8ef55c70d31.svg
Runji Lin
user
RunjiLin
6764dda159e2ef96ec013c26
Tianhao Li
6764dda159e2ef96ec013c27
Tingyu Xia
6764dda159e2ef96ec013c28
Xingzhang Ren
claimed_verified
2025-01-03T14:05:49.873Z
645c82c4081be4b32049633a
/avatars/e5a08cf0ec5a04bd9d66111382ce0508.svg
xzhren
user
xingzhang
6764dda159e2ef96ec013c29
Xuancheng Ren
6764dda159e2ef96ec013c2a
Yang Fan
6764dda159e2ef96ec013c2b
Yang Su
claimed_verified
2024-12-20T13:55:55.786Z
63481fea27bd5edd982328f9
https://cdn-avatars.huggingface.co/v1/production/uploads/1665670948064-noauth.jpeg
Yang Su
user
yang-su2000
6764dda159e2ef96ec013c2c
Yichang Zhang
6764dda159e2ef96ec013c2d
Yu Wan
6764dda159e2ef96ec013c2e
Yuqiong Liu
admin_assigned
2024-12-20T09:14:40.709Z
666aacfb918ba11c7c598194
/avatars/45bee8f1fdbdd256ee47d25e4bf01a7a.svg
Yuqiong Liu
user
lyq333
6764dda159e2ef96ec013c2f
Zeyu Cui
admin_assigned
2024-12-20T09:14:34.133Z
672c25ca8cfb61188128eb6f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/FJWy9Tt7UQmu9KcTOx3Rt.png
Zeyu Cui
user
misakamage
6764dda159e2ef96ec013c30
Zhenru Zhang
6764dda159e2ef96ec013c31
Zihan Qiu
admin_assigned
2024-12-20T08:41:53.155Z
647ccbd6e07cf9bb2d485244
/avatars/e8915abaff04f6762247e196b7cf84df.svg
Zihan Qiu
user
QwQZh
Qwen2.5 Technical Report
In this report, we introduce Qwen2.5, a comprehensive series of large
language models (LLMs) designed to meet diverse needs. Compared to previous
iterations, Qwen 2.5 has been significantly improved during both the
pre-training and post-training stages. In terms of pre-training, we have scaled
the high-quality pre-training datasets from the previous 7 trillion tokens to
18 trillion tokens. This provides a strong foundation for common sense, expert
knowledge, and reasoning capabilities. In terms of post-training, we implement
intricate supervised finetuning with over 1 million samples, as well as
multistage reinforcement learning. Post-training techniques enhance human
preference, and notably improve long text generation, structural data analysis,
and instruction following. To handle diverse and varied use cases effectively,
we present Qwen2.5 LLM series in rich sizes. Open-weight offerings include base
and instruction-tuned models, with quantized versions available. In addition,
for hosted solutions, the proprietary models currently include two
mixture-of-experts (MoE) variants: Qwen2.5-Turbo and Qwen2.5-Plus, both
available from Alibaba Cloud Model Studio. Qwen2.5 has demonstrated top-tier
performance on a wide range of benchmarks evaluating language understanding,
reasoning, mathematics, coding, human preference alignment, etc. Specifically,
the open-weight flagship Qwen2.5-72B-Instruct outperforms a number of open and
proprietary models and demonstrates competitive performance to the
state-of-the-art open-weight model, Llama-3-405B-Instruct, which is around 5
times larger. Qwen2.5-Turbo and Qwen2.5-Plus offer superior cost-effectiveness
while performing competitively against GPT-4o-mini and GPT-4o respectively.
Additionally, as the foundation, Qwen2.5 models have been instrumental in
training specialized models such as Qwen2.5-Math, Qwen2.5-Coder, QwQ, and
multimodal models.
Qwen2.5, an enhanced series of large language models, demonstrates superior performance across various benchmarks and use cases through extensive pre-training and advanced post-training techniques.
large language models pre-training post-training supervised finetuning reinforcement learning mixture-of-experts Qwen2.5-Turbo Qwen2.5-Plus language understanding reasoning mathematics coding human preference alignment Qwen2.5-72B-Instruct Llama-3-405B-Instruct GPT-4o-mini GPT-4o multimodal models