Babel: Open Multilingual Large Language Models Serving Over 90% of
  Global Speakers
Large language models (LLMs) have revolutionized natural language processing
(NLP), yet open-source multilingual LLMs remain scarce, with existing models
often limited in language coverage. Such models typically prioritize
well-resourced languages, while widely spoken but under-resourced languages are
often overlooked. To address this disparity, we introduce Babel, an
open multilingual LLM that covers the top 25 languages by number of speakers,
supports over 90% of the global population, and includes many languages
neglected by other open multilingual LLMs. Unlike traditional continue
pretraining approaches, Babel expands its parameter count through a layer
extension technique that elevates Babel's performance ceiling. We introduce two
variants: Babel-9B, designed for efficient inference and
fine-tuning, and Babel-83B, which sets a new standard for open
multilingual LLMs. Extensive evaluations on multilingual tasks demonstrate its
superior performance compared to open LLMs of comparable size. In addition,
using open-source supervised fine-tuning datasets, Babel achieves remarkable
performance, with Babel-9B-Chat leading among 10B-sized LLMs and Babel-83B-Chat
setting a new standard for multilingual tasks, reaching the same level of
commercial models.
67c666245e2443d7d5e9b76a
Yiran Zhao
claimed_verified
2025-03-04T08:51:21.231Z
64802face9ff472e30dc1ceb
https://cdn-avatars.huggingface.co/v1/production/uploads/64802face9ff472e30dc1ceb/bcwTlgpaUrU7m2RMB5zCc.png
Yiran Zhao
user
Yiran0924
67c666245e2443d7d5e9b76b
Chaoqun Liu
claimed_verified
2025-03-06T09:27:33.956Z
61657b0b20606e5e73f611cc
https://cdn-avatars.huggingface.co/v1/production/uploads/61657b0b20606e5e73f611cc/6ZPne2GYlWkxrx35ND1P8.png
CHAOQUN LIU
user
lukecq
67c666245e2443d7d5e9b76c
Yue Deng
67c666245e2443d7d5e9b76d
Jiahao Ying
admin_assigned
2025-03-06T09:35:39.926Z
671609f7664f44a151f1f0e8
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/fEQLuH1kdW5Pd9Y_J64hN.png
jiahao ying
user
jhying
67c666245e2443d7d5e9b76e
Mahani Aljunied
admin_assigned
2025-03-06T09:35:33.285Z
6539c87ba318a98bf0d15dd8
/avatars/beb9ba6eeacb61addc5897836bd59f55.svg
Mahani Aljunied
user
maljunied
67c666245e2443d7d5e9b76f
Zhaodonghui Li
67c666245e2443d7d5e9b770
Lidong Bing
admin_assigned
2025-03-06T09:35:19.611Z
6454685a548f22be598414c4
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/eMjMWKJ-AouF7eY1-RzGF.jpeg
Lidong Bing
user
LidongBing
67c666245e2443d7d5e9b771
Hou Pong Chan
admin_assigned
2025-03-06T09:35:50.272Z
604f67ef0fe8ff3ec13d71ef
https://cdn-avatars.huggingface.co/v1/production/uploads/604f67ef0fe8ff3ec13d71ef/KhUwWvZ3OJ9nEee3B-SXO.png
Hou Pong (Ken) Chan
user
kenchan0226
67c666245e2443d7d5e9b772
Yu Rong
67c666245e2443d7d5e9b773
Deli Zhao
67c666245e2443d7d5e9b774
Wenxuan Zhang
claimed_verified
2025-03-06T09:27:36.769Z
60dff6ae19a362a8c27862aa
https://cdn-avatars.huggingface.co/v1/production/uploads/60dff6ae19a362a8c27862aa/LIYzLB3cdPh-B3XIBgBCC.jpeg
Wenxuan Zhang
user
isakzhang
Babel: Open Multilingual Large Language Models Serving Over 90% of
  Global Speakers
Large language models (LLMs) have revolutionized natural language processing
(NLP), yet open-source multilingual LLMs remain scarce, with existing models
often limited in language coverage. Such models typically prioritize
well-resourced languages, while widely spoken but under-resourced languages are
often overlooked. To address this disparity, we introduce Babel, an
open multilingual LLM that covers the top 25 languages by number of speakers,
supports over 90% of the global population, and includes many languages
neglected by other open multilingual LLMs. Unlike traditional continue
pretraining approaches, Babel expands its parameter count through a layer
extension technique that elevates Babel's performance ceiling. We introduce two
variants: Babel-9B, designed for efficient inference and
fine-tuning, and Babel-83B, which sets a new standard for open
multilingual LLMs. Extensive evaluations on multilingual tasks demonstrate its
superior performance compared to open LLMs of comparable size. In addition,
using open-source supervised fine-tuning datasets, Babel achieves remarkable
performance, with Babel-9B-Chat leading among 10B-sized LLMs and Babel-83B-Chat
setting a new standard for multilingual tasks, reaching the same level of
commercial models.
multilingual LLMs natural language processing (NLP) large language models (LLMs) layer extension technique parameter count performance ceiling fine-tuning supervised fine-tuning datasets commercial models