SeaLLMs -- Large Language Models for Southeast Asia
Despite the remarkable achievements of large language models (LLMs) in
various tasks, there remains a linguistic bias that favors high-resource
languages, such as English, often at the expense of low-resource and regional
languages. To address this imbalance, we introduce SeaLLMs, an innovative
series of language models that specifically focuses on Southeast Asian (SEA)
languages. SeaLLMs are built upon the Llama-2 model and further advanced
through continued pre-training with an extended vocabulary, specialized
instruction and alignment tuning to better capture the intricacies of regional
languages. This allows them to respect and reflect local cultural norms,
customs, stylistic preferences, and legal considerations. Our comprehensive
evaluation demonstrates that SeaLLM-13b models exhibit superior performance
across a wide spectrum of linguistic tasks and assistant-style
instruction-following capabilities relative to comparable open-source models.
Moreover, they outperform ChatGPT-3.5 in non-Latin languages, such as Thai,
Khmer, Lao, and Burmese, by large margins while remaining lightweight and
cost-effective to operate.
656d317b8fb1ddf0d5524830
Xuan-Phi Nguyen
claimed_verified
2023-12-04T07:39:13.769Z
6393f04df7e70dd0166c004e
/avatars/acf4e9e0204a7ff7445aecc4102700cd.svg
Phi Nguyen
user
nxphi47
656d317b8fb1ddf0d5524831
Wenxuan Zhang
admin_assigned
2023-12-04T08:26:30.819Z
60dff6ae19a362a8c27862aa
https://cdn-avatars.huggingface.co/v1/production/uploads/60dff6ae19a362a8c27862aa/LIYzLB3cdPh-B3XIBgBCC.jpeg
Wenxuan Zhang
user
isakzhang
656d317b8fb1ddf0d5524832
Xin Li
claimed_verified
2023-12-05T08:41:36.038Z
63913b120cf6b11c487ca31d
/avatars/aec44edd5470dd6e767e0a25efd6fb5d.svg
Xin Li
user
lixin4ever
656d317b8fb1ddf0d5524833
Mahani Aljunied
admin_assigned
2023-12-04T08:27:24.831Z
6539c87ba318a98bf0d15dd8
/avatars/beb9ba6eeacb61addc5897836bd59f55.svg
Mahani Aljunied
user
maljunied
656d317b8fb1ddf0d5524834
Qingyu Tan
admin_assigned
2023-12-04T08:27:32.917Z
62bd46d35f59ff2320d7e31f
/avatars/19b5817a5fd531cca9ea83b76f46d7d3.svg
Qingyu Tan
user
tonytan48
656d317b8fb1ddf0d5524835
Liying Cheng
admin_assigned
2023-12-04T08:27:39.676Z
60be079e9418b3c3717a07ce
/avatars/2ae2822e2645f5e52be70b6f964feda3.svg
Liying Cheng
user
liyingnlp
656d317b8fb1ddf0d5524836
Guanzheng Chen
admin_assigned
2023-12-04T08:27:50.248Z
645475e2548f22be59847604
https://cdn-avatars.huggingface.co/v1/production/uploads/645475e2548f22be59847604/EhSurrZ25u31qQ2TVXQXt.jpeg
Chen
user
Guanzheng
656d317b8fb1ddf0d5524837
Yue Deng
656d317b8fb1ddf0d5524838
Sen Yang
admin_assigned
2023-12-04T08:30:32.743Z
638dd7d8a11654155baba973
/avatars/a95c01cd03d5da209b4d82d50aa01232.svg
Sen Yang
user
ringos
656d317b8fb1ddf0d5524839
Chaoqun Liu
admin_assigned
2023-12-04T08:30:39.598Z
61657b0b20606e5e73f611cc
https://cdn-avatars.huggingface.co/v1/production/uploads/61657b0b20606e5e73f611cc/6ZPne2GYlWkxrx35ND1P8.png
CHAOQUN LIU
user
lukecq
656d317b8fb1ddf0d552483a
Hang Zhang
admin_assigned
2023-12-04T08:31:00.170Z
638ee4bca26f3d5af37d7698
/avatars/36f1c81dd82379f863601a1767ff630a.svg
zhanghang
user
hangzhang-nlp
656d317b8fb1ddf0d552483b
Lidong Bing
admin_assigned
2023-12-04T08:31:06.774Z
6454685a548f22be598414c4
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/eMjMWKJ-AouF7eY1-RzGF.jpeg
Lidong Bing
user
LidongBing
SeaLLMs -- Large Language Models for Southeast Asia
Despite the remarkable achievements of large language models (LLMs) in
various tasks, there remains a linguistic bias that favors high-resource
languages, such as English, often at the expense of low-resource and regional
languages. To address this imbalance, we introduce SeaLLMs, an innovative
series of language models that specifically focuses on Southeast Asian (SEA)
languages. SeaLLMs are built upon the Llama-2 model and further advanced
through continued pre-training with an extended vocabulary, specialized
instruction and alignment tuning to better capture the intricacies of regional
languages. This allows them to respect and reflect local cultural norms,
customs, stylistic preferences, and legal considerations. Our comprehensive
evaluation demonstrates that SeaLLM-13b models exhibit superior performance
across a wide spectrum of linguistic tasks and assistant-style
instruction-following capabilities relative to comparable open-source models.
Moreover, they outperform ChatGPT-3.5 in non-Latin languages, such as Thai,
Khmer, Lao, and Burmese, by large margins while remaining lightweight and
cost-effective to operate.
SeaLLMs, a series of Llama-2 based models tailored for Southeast Asian languages, demonstrate superior performance across linguistic tasks and outperform ChatGPT-3.5 in non-Latin languages while being lightweight and cost-effective.
SeaLLMs Llama-2 pre-training vocabulary instruction tuning alignment tuning linguistic tasks assistant-style ChatGPT-3.5 Thai Khmer Lao Burmese