Orion-14B: Open-source Multilingual Large Language Models
In this study, we introduce Orion-14B, a collection of multilingual large
language models with 14 billion parameters. We utilize a data scheduling
approach to train a foundational model on a diverse corpus of 2.5 trillion
tokens, sourced from texts in English, Chinese, Japanese, Korean, and other
languages. Additionally, we fine-tuned a series of models tailored for
conversational applications and other specific use cases. Our evaluation
results demonstrate that Orion-14B achieves state-of-the-art performance across
a broad spectrum of tasks. We make the Orion-14B model family and its
associated code publicly accessible https://github.com/OrionStarAI/Orion,
aiming to inspire future research and practical applications in the field.
65b0738c9bd5ea963a85e8de
Du Chen
admin_assigned
2024-01-24T09:49:26.367Z
64c483deed521f27a4f7dab4
https://cdn-avatars.huggingface.co/v1/production/uploads/64c483deed521f27a4f7dab4/J9pKV-3-3ZbsqVPERIIko.png
chendulearn
user
chendulearn
65b0738c9bd5ea963a85e8df
Yi Huang
65b0738c9bd5ea963a85e8e0
Xiaopu Li
claimed_verified
2024-08-16T08:20:19.549Z
61b70ccd7b131aad98e4291c
/avatars/4c58c9bcedd54d67835cdfa4baba9778.svg
lee
user
sharp
65b0738c9bd5ea963a85e8e1
Yongqiang Li
admin_assigned
2024-01-24T09:52:13.967Z
63f32e9b0be81bdc5d8f6dd6
/avatars/1f790160a1199f2e2e0a4b8878a4ca97.svg
yongqiang li
user
yqli2420
65b0738c9bd5ea963a85e8e2
Yongqiang Liu
admin_assigned
2024-01-24T09:51:42.138Z
6540f519c81b3728f0a099db
/avatars/bd787cee9c5954e6746e4eaaa2151409.svg
yongqiang liu
user
liuyongq
65b0738c9bd5ea963a85e8e3
Haihui Pan
65b0738c9bd5ea963a85e8e4
Leichao Xu
admin_assigned
2024-01-24T09:52:39.465Z
643c26d9b409fef15e097937
/avatars/1fda8d21f1a479db59a30d184c8a8786.svg
xuleichao
user
leichao
65b0738c9bd5ea963a85e8e5
Dacheng Zhang
admin_assigned
2024-01-24T09:56:20.210Z
642401213fa01ecba6fd1e9b
/avatars/c36bce29c1738be9470bcbf084a7c09a.svg
ZhangDacheng
user
DachengZhang
65b0738c9bd5ea963a85e8e6
Zhipeng Zhang
65b0738c9bd5ea963a85e8e7
Kun Han
admin_assigned
2024-01-24T09:58:12.384Z
63ec4b3459a46fd5f4b699e3
/avatars/21d1b4df73b82ffd1b8d31e462daee70.svg
Kun Han
user
hankun11
Orion-14B: Open-source Multilingual Large Language Models
In this study, we introduce Orion-14B, a collection of multilingual large
language models with 14 billion parameters. We utilize a data scheduling
approach to train a foundational model on a diverse corpus of 2.5 trillion
tokens, sourced from texts in English, Chinese, Japanese, Korean, and other
languages. Additionally, we fine-tuned a series of models tailored for
conversational applications and other specific use cases. Our evaluation
results demonstrate that Orion-14B achieves state-of-the-art performance across
a broad spectrum of tasks. We make the Orion-14B model family and its
associated code publicly accessible https://github.com/OrionStarAI/Orion,
aiming to inspire future research and practical applications in the field.
A multilingual large language model with 14 billion parameters achieves state-of-the-art performance across various tasks through data scheduling and fine-tuning.
multilingual large language models data scheduling conversational applications state-of-the-art performance