A Technical Report for Polyglot-Ko: Open-Source Large-Scale Korean
  Language Models
Polyglot is a pioneering project aimed at enhancing the non-English language
performance of multilingual language models. Despite the availability of
various multilingual models such as mBERT (Devlin et al., 2019), XGLM (Lin et
al., 2022), and BLOOM (Scao et al., 2022), researchers and developers often
resort to building monolingual models in their respective languages due to the
dissatisfaction with the current multilingual models non-English language
capabilities. Addressing this gap, we seek to develop advanced multilingual
language models that offer improved performance in non-English languages. In
this paper, we introduce the Polyglot Korean models, which represent a specific
focus rather than being multilingual in nature. In collaboration with TUNiB,
our team collected 1.2TB of Korean data meticulously curated for our research
journey. We made a deliberate decision to prioritize the development of Korean
models before venturing into multilingual models. This choice was motivated by
multiple factors: firstly, the Korean models facilitated performance
comparisons with existing multilingual models; and finally, they catered to the
specific needs of Korean companies and researchers. This paper presents our
work in developing the Polyglot Korean models, which propose some steps towards
addressing the non-English language performance gap in multilingual language
models.
647eaa82becb41a2729772cd
Hyunwoong Ko
claimed_verified
2023-06-07T19:41:00.982Z
5fd888cf61e46993190ce543
https://cdn-avatars.huggingface.co/v1/production/uploads/1634604273263-5fd888cf61e46993190ce543.jpeg
Hyunwoong Ko
user
hyunwoongko
647eaa82becb41a2729772ce
Kichang Yang
admin_assigned
2023-06-06T08:49:30.533Z
5fa11101a13e063b8b2b5d34
https://cdn-avatars.huggingface.co/v1/production/uploads/5fa11101a13e063b8b2b5d34/p2Sm4b64hn_xXg1GtrkIA.jpeg
Kichang Yang
user
jason9693
647eaa82becb41a2729772cf
Minho Ryu
admin_assigned
2023-06-06T08:50:04.739Z
60436d159e905013ae8715d7
https://cdn-avatars.huggingface.co/v1/production/uploads/1623809612769-60436d159e905013ae8715d7.jpeg
Minho Ryu
user
bzantium
647eaa82becb41a2729772d0
Taekyoon Choi
647eaa82becb41a2729772d1
Seungmu Yang
claimed_verified
2023-06-07T10:09:40.241Z
61e78b9e68f7ae955e4b0b8e
/avatars/ecb4c6db9092c0ba84695cf0a33e5a91.svg
Seungmoo Yang
user
Moo
647eaa82becb41a2729772d2
jiwung Hyun
admin_assigned
2023-06-06T08:52:26.750Z
61e78aa282b19b93e1a53cbe
https://cdn-avatars.huggingface.co/v1/production/uploads/1662805242568-61e78aa282b19b93e1a53cbe.jpeg
Jiwung Hyun
user
kabbi
647eaa82becb41a2729772d3
Sungho Park
admin_assigned
2023-06-06T08:53:18.455Z
625fc71affe8827cb1d3b1f8
https://cdn-avatars.huggingface.co/v1/production/uploads/1660573263107-625fc71affe8827cb1d3b1f8.png
Sungho Park
user
naem1023
A Technical Report for Polyglot-Ko: Open-Source Large-Scale Korean
  Language Models
Polyglot is a pioneering project aimed at enhancing the non-English language
performance of multilingual language models. Despite the availability of
various multilingual models such as mBERT (Devlin et al., 2019), XGLM (Lin et
al., 2022), and BLOOM (Scao et al., 2022), researchers and developers often
resort to building monolingual models in their respective languages due to the
dissatisfaction with the current multilingual models non-English language
capabilities. Addressing this gap, we seek to develop advanced multilingual
language models that offer improved performance in non-English languages. In
this paper, we introduce the Polyglot Korean models, which represent a specific
focus rather than being multilingual in nature. In collaboration with TUNiB,
our team collected 1.2TB of Korean data meticulously curated for our research
journey. We made a deliberate decision to prioritize the development of Korean
models before venturing into multilingual models. This choice was motivated by
multiple factors: firstly, the Korean models facilitated performance
comparisons with existing multilingual models; and finally, they catered to the
specific needs of Korean companies and researchers. This paper presents our
work in developing the Polyglot Korean models, which propose some steps towards
addressing the non-English language performance gap in multilingual language
models.
Polyglot focuses on developing specialized Korean language models to improve non-English language performance in multilingual models by leveraging a large dataset.
multilingual language models mBERT XGLM BLOOM Korean models dataset language performance