Aya 23: Open Weight Releases to Further Multilingual Progress
This technical report introduces Aya 23, a family of multilingual language
models. Aya 23 builds on the recent release of the Aya model (\"Ust\"un et al.,
2024), focusing on pairing a highly performant pre-trained model with the
recently released Aya collection (Singh et al., 2024). The result is a powerful
multilingual large language model serving 23 languages, expanding state-of-art
language modeling capabilities to approximately half of the world's population.
The Aya model covered 101 languages whereas Aya 23 is an experiment in depth vs
breadth, exploring the impact of allocating more capacity to fewer languages
that are included during pre-training. Aya 23 outperforms both previous
massively multilingual models like Aya 101 for the languages it covers, as well
as widely used models like Gemma, Mistral and Mixtral on an extensive range of
discriminative and generative tasks. We release the open weights for both the
8B and 35B models as part of our continued commitment for expanding access to
multilingual progress.
6653ed0583fac8ccdc09c6fd
Viraat Aryabumi
extracted_confirmed
2024-05-27T04:13:50.260Z
633e9b78acd351fc37d69560
https://cdn-avatars.huggingface.co/v1/production/uploads/633e9b78acd351fc37d69560/rCk8z-C4oOlrksp-QE2lV.jpeg
Viraat Aryabumi
user
viraat
6653ed0583fac8ccdc09c6fe
John Dang
admin_assigned
2024-05-27T08:49:56.573Z
65c581dfc3fa039f843991f6
https://cdn-avatars.huggingface.co/v1/production/uploads/65c581dfc3fa039f843991f6/O4ywGgQQrzps5JYeR3o0y.jpeg
John Dang
user
johndang-cohere
6653ed0583fac8ccdc09c6ff
Dwarak Talupuru
6653ed0583fac8ccdc09c700
Saurabh Dash
admin_assigned
2024-05-27T08:50:22.027Z
63adeb79a31e9ea8298ad4ac
https://cdn-avatars.huggingface.co/v1/production/uploads/63adeb79a31e9ea8298ad4ac/a_kEJXPLkfaHDUN3XcTYA.jpeg
Saurabh Dash
user
saurabhdash
6653ed0583fac8ccdc09c701
David Cairuz
admin_assigned
2024-05-27T08:52:57.723Z
64674f1a8334813a7ae33023
https://cdn-avatars.huggingface.co/v1/production/uploads/64674f1a8334813a7ae33023/VjxQYzaXW7zlH7ktPBsEX.jpeg
David Cairuz
user
davidcairuz
6653ed0583fac8ccdc09c702
Hangyu Lin
6653ed0583fac8ccdc09c703
Bharat Venkitesh
admin_assigned
2024-05-27T08:53:51.118Z
65e8e96868cd34406483a639
/avatars/77fbe9ebd1fccd2e15abb403e2139652.svg
Bharat venkitesh
user
bharatv007
6653ed0583fac8ccdc09c704
Madeline Smith
admin_assigned
2024-05-27T08:53:58.583Z
6351689040dffad651f0d26b
/avatars/65808f9bf9243c7e8bf7130cdf86166b.svg
Madeline Smith
user
madelzzz
6653ed0583fac8ccdc09c705
Kelly Marchisio
admin_assigned
2024-05-27T08:54:17.273Z
63bc6313d8d676a229a2b9c8
/avatars/d11fa232a03d5d0ac88c177ef1cf187d.svg
Kelly Marchisio
user
kellymarchisio
6653ed0583fac8ccdc09c706
Sebastian Ruder
admin_assigned
2024-05-27T08:54:26.270Z
624575789dc26c59bae6a19e
https://cdn-avatars.huggingface.co/v1/production/uploads/1648719221694-noauth.jpeg
Sebastian Ruder
user
ruder
6653ed0583fac8ccdc09c707
Acyr Locatelli
admin_assigned
2024-05-27T08:54:35.114Z
60d370b613f774189902f556
/avatars/717bd7955f30eb76310d2074fd51afce.svg
Acyr Locatelli
user
acyrl
6653ed0583fac8ccdc09c708
Julia Kreutzer
admin_assigned
2024-05-27T08:54:49.294Z
6544e43b12da508864c38f96
/avatars/76f0cd55b4bf9c03d2686e146c6f795f.svg
Julia Kreutzer
user
JuliaKreutzerCohere
6653ed0583fac8ccdc09c709
Nick Frosst
admin_assigned
2024-05-27T08:54:57.514Z
6402473006c715b934011e0d
/avatars/e312516c1e0edbf65418b5285b3164d4.svg
nick frosst
user
nickfrosst
6653ed0583fac8ccdc09c70a
Phil Blunsom
6653ed0583fac8ccdc09c70b
Marzieh Fadaee
admin_assigned
2024-05-27T08:55:08.278Z
6441042d5d600fb0951a5f99
https://cdn-avatars.huggingface.co/v1/production/uploads/6441042d5d600fb0951a5f99/4CbOaYcEz99BtVAQvnGTn.jpeg
Marzieh Fadaee
user
MarziehFadaee
6653ed0583fac8ccdc09c70c
Ahmet Üstün
admin_assigned
2024-05-27T08:55:16.611Z
60d35d7ad7b174177faabd5b
/avatars/4e5403b9d4a845a2d21e8217dc3c16d2.svg
Ahmet Üstün
user
ahmetu
6653ed0583fac8ccdc09c70d
Sara Hooker
extracted_confirmed
2024-05-27T03:43:46.321Z
63434eb76f59b79da07dbddf
https://cdn-avatars.huggingface.co/v1/production/uploads/63434eb76f59b79da07dbddf/BEwmVjqPNYlqmutXG0G6e.jpeg
Sara Hooker
user
sarahooker
Aya 23: Open Weight Releases to Further Multilingual Progress
This technical report introduces Aya 23, a family of multilingual language
models. Aya 23 builds on the recent release of the Aya model (\"Ust\"un et al.,
2024), focusing on pairing a highly performant pre-trained model with the
recently released Aya collection (Singh et al., 2024). The result is a powerful
multilingual large language model serving 23 languages, expanding state-of-art
language modeling capabilities to approximately half of the world's population.
The Aya model covered 101 languages whereas Aya 23 is an experiment in depth vs
breadth, exploring the impact of allocating more capacity to fewer languages
that are included during pre-training. Aya 23 outperforms both previous
massively multilingual models like Aya 101 for the languages it covers, as well
as widely used models like Gemma, Mistral and Mixtral on an extensive range of
discriminative and generative tasks. We release the open weights for both the
8B and 35B models as part of our continued commitment for expanding access to
multilingual progress.
Aya 23 is a powerful multilingual language model for 23 languages, outperforming previous models on a wide range of tasks.
multilingual language models pre-trained model Aya model Aya collection state-of-art language modeling Gemma Mistral Mixtral discriminative tasks generative tasks