Aya Model: An Instruction Finetuned Open-Access Multilingual Language
  Model
Recent breakthroughs in large language models (LLMs) have centered around a
handful of data-rich languages. What does it take to broaden access to
breakthroughs beyond first-class citizen languages? Our work introduces Aya, a
massively multilingual generative language model that follows instructions in
101 languages of which over 50% are considered as lower-resourced. Aya
outperforms mT0 and BLOOMZ on the majority of tasks while covering double the
number of languages. We introduce extensive new evaluation suites that broaden
the state-of-art for multilingual eval across 99 languages -- including
discriminative and generative tasks, human evaluation, and simulated win rates
that cover both held-out tasks and in-distribution performance. Furthermore, we
conduct detailed investigations on the optimal finetuning mixture composition,
data pruning, as well as the toxicity, bias, and safety of our models. We
open-source our instruction datasets and our model at
https://hf.co/CohereForAI/aya-101
65cae379d5822659eced5572
Ahmet Üstün
admin_assigned
2024-02-13T08:24:42.414Z
60d35d7ad7b174177faabd5b
/avatars/4e5403b9d4a845a2d21e8217dc3c16d2.svg
Ahmet Üstün
user
ahmetu
65cae379d5822659eced5573
Viraat Aryabumi
admin_assigned
2024-02-13T08:24:48.751Z
633e9b78acd351fc37d69560
https://cdn-avatars.huggingface.co/v1/production/uploads/633e9b78acd351fc37d69560/rCk8z-C4oOlrksp-QE2lV.jpeg
Viraat Aryabumi
user
viraat
65cae379d5822659eced5574
Zheng-Xin Yong
admin_assigned
2024-02-13T08:25:48.057Z
61424bf4f0d914a5f606a823
https://cdn-avatars.huggingface.co/v1/production/uploads/61424bf4f0d914a5f606a823/0td8lR4elBaVvJUD9Pojh.png
Yong Zheng-Xin
user
yongzx
65cae379d5822659eced5575
Wei-Yin Ko
admin_assigned
2024-02-13T08:25:54.888Z
640d286c9e5247967ff69840
/avatars/01b39240236e41859f7a9fd71f83382a.svg
Wei-Yin Ko
user
weiyinko
65cae379d5822659eced5576
Daniel D'souza
admin_assigned
2024-02-13T08:26:14.368Z
657495cbec3bf96e433489a2
/avatars/3d0f05abaa8c87d37d06521a19d41aee.svg
Daniel Dsouza
user
DanielDsouza
65cae379d5822659eced5577
Gbemileke Onilude
65cae379d5822659eced5578
Neel Bhandari
admin_assigned
2024-02-13T08:27:00.364Z
639ece4c0d679f53942c9c23
/avatars/e253a796a6179fde28323bc3d84ebb90.svg
Neel Bhandari
user
neelbhandari
65cae379d5822659eced5579
Shivalika Singh
admin_assigned
2024-02-13T08:27:07.160Z
62c2175d756039dd0dd20509
https://cdn-avatars.huggingface.co/v1/production/uploads/1677753847837-62c2175d756039dd0dd20509.jpeg
Shivalika Singh
user
shivalikasingh
65cae379d5822659eced557a
Hui-Lee Ooi
65cae379d5822659eced557b
Amr Kayid
admin_assigned
2024-02-13T08:27:18.115Z
60099e06168be9b5445234b9
https://cdn-avatars.huggingface.co/v1/production/uploads/60099e06168be9b5445234b9/l-l9xmtfzhMPAUynFUNsa.png
Amr Kayid
user
AmrMKayid
65cae379d5822659eced557c
Freddie Vargus
claimed_verified
2024-02-13T08:10:10.780Z
618cb927ec9a3fd349fa7e26
/avatars/75c81a484fcafac3b40ed6ca01832faf.svg
Freddie Vargus
user
freddie
65cae379d5822659eced557d
Phil Blunsom
65cae379d5822659eced557e
Shayne Longpre
admin_assigned
2024-02-13T08:27:30.520Z
61f4283a81c4d30f58140242
/avatars/a1cf1ef1fd442c36ed65c68e51919fed.svg
Shayne Longpre
user
Shayne
65cae379d5822659eced557f
Niklas Muennighoff
admin_assigned
2024-02-13T08:27:37.915Z
5f1eb362eec0ad2a071ad6e2
https://cdn-avatars.huggingface.co/v1/production/uploads/5f1eb362eec0ad2a071ad6e2/IXMYkYKuTwn6kBdWnQeeY.png
Niklas Muennighoff
user
Muennighoff
65cae379d5822659eced5580
Marzieh Fadaee
admin_assigned
2024-02-13T08:27:43.590Z
6441042d5d600fb0951a5f99
https://cdn-avatars.huggingface.co/v1/production/uploads/6441042d5d600fb0951a5f99/4CbOaYcEz99BtVAQvnGTn.jpeg
Marzieh Fadaee
user
MarziehFadaee
65cae379d5822659eced5581
Julia Kreutzer
admin_assigned
2024-02-13T08:27:58.197Z
6544e43b12da508864c38f96
/avatars/76f0cd55b4bf9c03d2686e146c6f795f.svg
Julia Kreutzer
user
JuliaKreutzerCohere
65cae379d5822659eced5582
Sara Hooker
extracted_confirmed
2024-05-27T03:43:44.600Z
63434eb76f59b79da07dbddf
https://cdn-avatars.huggingface.co/v1/production/uploads/63434eb76f59b79da07dbddf/BEwmVjqPNYlqmutXG0G6e.jpeg
Sara Hooker
user
sarahooker
Aya Model: An Instruction Finetuned Open-Access Multilingual Language
  Model
Recent breakthroughs in large language models (LLMs) have centered around a
handful of data-rich languages. What does it take to broaden access to
breakthroughs beyond first-class citizen languages? Our work introduces Aya, a
massively multilingual generative language model that follows instructions in
101 languages of which over 50% are considered as lower-resourced. Aya
outperforms mT0 and BLOOMZ on the majority of tasks while covering double the
number of languages. We introduce extensive new evaluation suites that broaden
the state-of-art for multilingual eval across 99 languages -- including
discriminative and generative tasks, human evaluation, and simulated win rates
that cover both held-out tasks and in-distribution performance. Furthermore, we
conduct detailed investigations on the optimal finetuning mixture composition,
data pruning, as well as the toxicity, bias, and safety of our models. We
open-source our instruction datasets and our model at
https://hf.co/CohereForAI/aya-101
Aya, a multilingual generative language model supporting over 50% lower-resourced languages, excels in both generative and discriminative tasks across 99 languages, and provides extensive evaluations and open-source resources.
large language models LLMs massively multilingual generative language model mT0 BLOOMZ instruction datasets multilingual eval discriminative tasks generative tasks human evaluation simulated win rates in-distribution performance finetuning mixture composition data pruning toxicity bias safety