InkubaLM: A small language model for low-resource African languages
High-resource language models often fall short in the African context, where
there is a critical need for models that are efficient, accessible, and locally
relevant, even amidst significant computing and data constraints. This paper
introduces InkubaLM, a small language model with 0.4 billion parameters, which
achieves performance comparable to models with significantly larger parameter
counts and more extensive training data on tasks such as machine translation,
question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM
outperforms many larger models in sentiment analysis and demonstrates
remarkable consistency across multiple languages. This work represents a
pivotal advancement in challenging the conventional paradigm that effective
language models must rely on substantial resources. Our model and datasets are
publicly available \url{https://huggingface.co/lelapa} to encourage
research and development on low-resource languages.
66d585b72e0412fa2abab646
Atnafu Lambebo Tonja
admin_assigned
2024-09-02T10:00:20.105Z
6245232cf59b8d262df30db6
https://cdn-avatars.huggingface.co/v1/production/uploads/6245232cf59b8d262df30db6/zmQuv1vqGL3MsCAnzGm9y.jpeg
Atnafu Lambebo Tonja
user
Atnafu
66d585b72e0412fa2abab647
Bonaventure F. P. Dossou
admin_assigned
2024-09-02T10:00:29.734Z
605dd96184a51781b143a131
https://cdn-avatars.huggingface.co/v1/production/uploads/605dd96184a51781b143a131/wBUY_uAcQOWdRW21uZLS7.jpeg
Bonaventure Dossou
user
bonadossou
66d585b72e0412fa2abab648
Jessica Ojo
admin_assigned
2024-09-02T10:00:35.872Z
650030261e170b85743bf3ca
https://cdn-avatars.huggingface.co/v1/production/uploads/650030261e170b85743bf3ca/3iq_cmj7wq03OgAR0CTtE.jpeg
Jessica O. Ojo
user
JessicaOjo
66d585b72e0412fa2abab649
Jenalea Rajab
admin_assigned
2024-09-02T10:00:41.768Z
64490668638bfa3ce138f373
/avatars/7db8f63634959927443925f49368ed5e.svg
Jenalea Rajab
user
Jenalea
66d585b72e0412fa2abab64a
Fadel Thior
66d585b72e0412fa2abab64b
Eric Peter Wairagala
admin_assigned
2024-09-02T10:00:51.410Z
6151c7140d6899cb526dbd96
/avatars/1fc151e025839e92af0beacd5a68ebff.svg
Eric Peter Wairagala
user
EricPeter
66d585b72e0412fa2abab64c
Aremu Anuoluwapo
admin_assigned
2024-09-02T10:00:58.957Z
628a9a8b6a2a449b99aa1969
https://cdn-avatars.huggingface.co/v1/production/uploads/1653342087811-628a9a8b6a2a449b99aa1969.jpeg
Aremu Anuoluwapo
user
aremuadeolajr
66d585b72e0412fa2abab64d
Pelonomi Moiloa
admin_assigned
2024-09-02T10:01:03.996Z
6481b502ec65b8b77d85bca5
https://cdn-avatars.huggingface.co/v1/production/uploads/6481b502ec65b8b77d85bca5/D0AwZ-9znmUHb-myzHL3M.jpeg
Pelonomi Moiloa
user
pelonomi
66d585b72e0412fa2abab64e
Jade Abbott
admin_assigned
2024-09-02T10:01:09.828Z
5fbf94dcf6c5c10afc0c9e09
https://cdn-avatars.huggingface.co/v1/production/uploads/1606390935561-noauth.jpeg
Jade Abbott
user
alienelf
66d585b72e0412fa2abab64f
Vukosi Marivate
admin_assigned
2024-09-02T10:01:15.372Z
617a68fef53050061a7294ea
https://cdn-avatars.huggingface.co/v1/production/uploads/617a68fef53050061a7294ea/J9B1gZ9CUPRMahP6QDSPL.jpeg
Vukosi Marivate
user
vukosi
66d585b72e0412fa2abab650
Benjamin Rosman
InkubaLM: A small language model for low-resource African languages
High-resource language models often fall short in the African context, where
there is a critical need for models that are efficient, accessible, and locally
relevant, even amidst significant computing and data constraints. This paper
introduces InkubaLM, a small language model with 0.4 billion parameters, which
achieves performance comparable to models with significantly larger parameter
counts and more extensive training data on tasks such as machine translation,
question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM
outperforms many larger models in sentiment analysis and demonstrates
remarkable consistency across multiple languages. This work represents a
pivotal advancement in challenging the conventional paradigm that effective
language models must rely on substantial resources. Our model and datasets are
publicly available \url{https://huggingface.co/lelapa} to encourage
research and development on low-resource languages.
small language model InkubaLM 0.4 billion parameters machine translation question-answering AfriMMLU AfriXnli sentiment analysis low-resource languages