Baichuan 2: Open Large-scale Language Models
Large language models (LLMs) have demonstrated remarkable performance on a
variety of natural language tasks based on just a few examples of natural
language instructions, reducing the need for extensive feature engineering.
However, most powerful LLMs are closed-source or limited in their capability
for languages other than English. In this technical report, we present Baichuan
2, a series of large-scale multilingual language models containing 7 billion
and 13 billion parameters, trained from scratch, on 2.6 trillion tokens.
Baichuan 2 matches or outperforms other open-source models of similar size on
public benchmarks like MMLU, CMMLU, GSM8K, and HumanEval. Furthermore, Baichuan
2 excels in vertical domains such as medicine and law. We will release all
pre-training model checkpoints to benefit the research community in better
understanding the training dynamics of Baichuan 2.
650a4a35ce81b73e63833df6
Aiyuan Yang
650a4a35ce81b73e63833df7
Bin Xiao
admin_assigned
2023-09-20T11:47:08.804Z
64f1568afcc7700c949c6154
/avatars/f23c1922f18731b77679b61aa89b309d.svg
BinXiao
user
BinXiao
650a4a35ce81b73e63833df8
Bingning Wang
650a4a35ce81b73e63833df9
Borong Zhang
650a4a35ce81b73e63833dfa
Chao Yin
650a4a35ce81b73e63833dfb
Chenxu Lv
650a4a35ce81b73e63833dfc
Da Pan
650a4a35ce81b73e63833dfd
Dian Wang
admin_assigned
2023-09-20T11:48:47.942Z
64a383eac9b4db7cc77b55af
/avatars/b8bb4757c1693aedb86aa763e3a66354.svg
Hellen Wang
user
wangdianhellen
650a4a35ce81b73e63833dfe
Dong Yan
650a4a35ce81b73e63833dff
Fan Yang
claimed_verified
2024-05-02T08:17:18.155Z
641c45c921964f8f6d451d16
/avatars/da06cc603f8f9ee46ddb7dc72aae5bec.svg
FanYang
user
fairyang
650a4a35ce81b73e63833e00
Fei Deng
650a4a35ce81b73e63833e01
Feng Wang
650a4a35ce81b73e63833e02
Feng Liu
650a4a35ce81b73e63833e03
Guangwei Ai
650a4a35ce81b73e63833e04
Guosheng Dong Haizhou Zhao
650a4a35ce81b73e63833e05
Hang Xu
650a4a35ce81b73e63833e06
Haoze Sun
claimed_verified
2025-01-23T15:05:54.056Z
6436bb0dd58a5ea528c55acb
/avatars/df17b66780e14e07bbe4625f068a94ad.svg
Alvin Sun
user
AlvinSunYooo
650a4a35ce81b73e63833e07
Hongda Zhang
admin_assigned
2023-09-20T12:22:42.147Z
642ae59bd651bae3c11e9371
/avatars/898275d5680a1a1f917f53448dfeedb3.svg
zhang hongda
user
hongdaaaaaaaa
650a4a35ce81b73e63833e08
Hui Liu
650a4a35ce81b73e63833e09
Jiaming Ji
admin_assigned
2023-09-20T12:24:19.245Z
6429b13816d4d8293c97a949
https://cdn-avatars.huggingface.co/v1/production/uploads/6429b13816d4d8293c97a949/LNuQ1gvW2NMmWyIuEpzCA.jpeg
JiJiaming
user
jijiaming
650a4a35ce81b73e63833e0a
Jian Xie
admin_assigned
2023-09-20T12:24:52.729Z
62d65139667051e0a29bffe7
/avatars/0252aa2bcd4cf1c8e4b87e5f164b6da5.svg
Jian Xie
user
hsaest
650a4a35ce81b73e63833e0b
Juntao Dai
admin_assigned
2023-09-20T12:25:01.971Z
6426a75336ab74ff2bca0542
https://cdn-avatars.huggingface.co/v1/production/uploads/6426a75336ab74ff2bca0542/qL3QQofTr4yo8Cs58oYwK.png
Juntao Dai
user
calico-1226
650a4a35ce81b73e63833e0c
Kun Fang
admin_assigned
2023-09-20T12:25:11.794Z
64e48059b9455cd53e17fe0c
/avatars/37863f1c106215a15997b3090ce53ab1.svg
Kun Fang
user
KunFang
650a4a35ce81b73e63833e0d
Lei Su Liang Song
650a4a35ce81b73e63833e0e
Lifeng Liu
admin_assigned
2023-09-20T12:26:02.714Z
6466db5b14e059dde8bbc0a2
/avatars/daf6aea9ded455c0224002090be904e3.svg
lili
user
liulifeng
650a4a35ce81b73e63833e0f
Liyun Ru
650a4a35ce81b73e63833e10
Luyao Ma
admin_assigned
2023-09-20T12:26:21.707Z
63f8b05ebb02399771116965
/avatars/edd1227a179d8f0db45b49ca53aeb4e7.svg
Luyao Ma
user
leonma
650a4a35ce81b73e63833e11
Mang Wang
650a4a35ce81b73e63833e12
Mickel Liu
admin_assigned
2023-09-20T12:26:32.951Z
643f986a18afbc4d1f3e2456
/avatars/96a6e5f18386a9864dfdded8f25ec72d.svg
Mickel Liu
user
mickelliu
650a4a35ce81b73e63833e13
MingAn Lin
650a4a35ce81b73e63833e14
Nuolan Nie
650a4a35ce81b73e63833e15
Peidong Guo
admin_assigned
2023-09-20T12:27:01.597Z
6487e2e1eec01aee99cf4c10
https://cdn-avatars.huggingface.co/v1/production/uploads/6487e2e1eec01aee99cf4c10/1U6zZ2OaUOrR1ueD5yraR.jpeg
Peidong Guo
user
GuoPD
650a4a35ce81b73e63833e16
Ruiyang Sun
admin_assigned
2023-09-20T12:28:51.811Z
63fca6b85d2bea4588bc9a94
/avatars/c8ae8c02d6d02af90678e8ee4122eafc.svg
Ruiyang Sun
user
RuiyangSun
650a4a35ce81b73e63833e17
Tao Zhang
650a4a35ce81b73e63833e18
Tianpeng Li
admin_assigned
2023-09-20T12:31:16.763Z
64be4c97ef8c0e42bf448469
/avatars/a2a45b5f17d3e27c59547312bcf7bb52.svg
Tianpeng Li
user
TJU-Tianpengli
650a4a35ce81b73e63833e19
Tianyu Li
650a4a35ce81b73e63833e1a
Wei Cheng
650a4a35ce81b73e63833e1b
Weipeng Chen
admin_assigned
2023-09-20T12:33:43.548Z
60d0870f778bafd0fbcfa3fa
https://cdn-avatars.huggingface.co/v1/production/uploads/1624280312404-60d0870f778bafd0fbcfa3fa.jpeg
Chen
user
Weipeng
650a4a35ce81b73e63833e1c
Xiangrong Zeng
650a4a35ce81b73e63833e1d
Xiaochuan Wang
admin_assigned
2023-09-20T12:34:58.345Z
6425449ace70775b51aecc28
/avatars/32609ac321cb0a98ace3b35e8a5e61cc.svg
5555
user
wangxiaochuan
650a4a35ce81b73e63833e1e
Xiaoxi Chen
admin_assigned
2023-09-20T12:35:16.087Z
646b2092e96a751c52505f3b
/avatars/4b3e93c9603ebe9822a2aa5b5e39d7b1.svg
xiaoxi
user
chenxiaoxi
650a4a35ce81b73e63833e1f
Xin Men
2023-09-20T12:35:30.937Z
650a4a35ce81b73e63833e20
Xin Yu
650a4a35ce81b73e63833e21
Xuehai Pan
admin_assigned
2023-09-20T12:36:25.424Z
64364e87fae2870051496e13
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/t67EsNoRvRYXKwi0G59oa.jpeg
Xuehai Pan
user
XuehaiPan
650a4a35ce81b73e63833e22
Yanjun Shen
admin_assigned
2023-09-20T12:36:38.322Z
64ced7c01720c7a483d0862f
/avatars/1fae685b7226a0a1c95b2d4993f2ab60.svg
shenyanjun
user
zilchshen
650a4a35ce81b73e63833e23
Yiding Wang
650a4a35ce81b73e63833e24
Yiyu Li
2023-09-20T12:36:56.644Z
650a4a35ce81b73e63833e25
Youxin Jiang
650a4a35ce81b73e63833e26
Yuchen Gao
admin_assigned
2023-09-20T12:37:14.423Z
6437d48151c7ebfc813c96b4
/avatars/d5ec60bb8247f1bb9b615382400bc462.svg
yuchen gao
user
mervyngo
650a4a35ce81b73e63833e27
Yupeng Zhang
650a4a35ce81b73e63833e28
Zenan Zhou
admin_assigned
2023-09-20T12:37:53.074Z
64020919f98fbc64bcdab85d
/avatars/530cfe3c16b6bd19e9a7232405e1918a.svg
zhouzenan
user
zhouzn
650a4a35ce81b73e63833e29
Zhiying Wu
admin_assigned
2023-09-20T12:15:14.603Z
646c36d1942fd7c135df8ff0
/avatars/9c85c6975e097d7de28538cb6fa091ff.svg
Zhiying Wu
user
wzy199520
Baichuan 2: Open Large-scale Language Models
Large language models (LLMs) have demonstrated remarkable performance on a
variety of natural language tasks based on just a few examples of natural
language instructions, reducing the need for extensive feature engineering.
However, most powerful LLMs are closed-source or limited in their capability
for languages other than English. In this technical report, we present Baichuan
2, a series of large-scale multilingual language models containing 7 billion
and 13 billion parameters, trained from scratch, on 2.6 trillion tokens.
Baichuan 2 matches or outperforms other open-source models of similar size on
public benchmarks like MMLU, CMMLU, GSM8K, and HumanEval. Furthermore, Baichuan
2 excels in vertical domains such as medicine and law. We will release all
pre-training model checkpoints to benefit the research community in better
understanding the training dynamics of Baichuan 2.
Baichuan 2, a series of 7 billion and 13 billion parameter multilingual LLMs, achieves competitive performance on benchmarks and excels in specialized domains, with open-source pre-training checkpoints released.
large language models multilingual language models pre-training MMLU CMMLU GSM8K HumanEval medicine law training dynamics