Kuwain 1.5B: An Arabic SLM via Language Injection
Enhancing existing models with new knowledge is a crucial aspect of AI
development. This paper introduces a novel method for integrating a new
language into a large language model (LLM). Our approach successfully
incorporates a previously unseen target language into an existing LLM without
compromising its prior knowledge. We trained a tiny model with 1.5 billion
parameters named Kuwain by injecting the Arabic language into a small
open-source model mainly trained in English. Our method demonstrates
significant improvements in Arabic language performance, with an average 8%
improvement across various benchmarks, while retaining the model's existing
knowledge with a minimum amount of the original model's data. This offers a
cost-effective alternative to training a comprehensive model in both English
and Arabic. The results highlight the potential for efficient, targeted
language model expansion without extensive retraining or resource-intensive
processes.
680733cf7722bb6407ca0787
Khalil Hennara
extracted_confirmed
2025-04-22T09:37:47.479Z
65276c7911a8a521c91bc10f
https://cdn-avatars.huggingface.co/v1/production/uploads/65276c7911a8a521c91bc10f/39dbuUtqQTJERTJ_WkxSh.jpeg
Khalil Hennara
user
Hennara
680733cf7722bb6407ca0788
Sara Chrouf
admin_assigned
2025-04-23T13:01:31.400Z
650b64fb443b2a59d4d432e3
/avatars/12f284e007e8b5cddeadf4491a0a1fe8.svg
Sara Chrouf
user
SaraChrouf
680733cf7722bb6407ca0789
Mohamed Motaism Hamed
claimed_verified
2025-04-23T08:37:25.702Z
63aa7667769a10efc404fbbc
https://cdn-avatars.huggingface.co/v1/production/uploads/63aa7667769a10efc404fbbc/tn8ZxUmTEMS0Gze7_F7JL.jpeg
Mohamed Motasim Hamed
user
Moatasem444
680733cf7722bb6407ca078a
Zeina Aldallal
extracted_confirmed
2025-05-04T10:24:16.300Z
65704741e1cfce1764ce652e
/avatars/9189aaf417426af4ebe381ed364a6c0e.svg
Zeina Aldallal
user
ZeinaD
680733cf7722bb6407ca078b
Omar Hadid
admin_assigned
2025-04-23T13:01:37.791Z
65aab3940150f64adfcf5ba6
https://cdn-avatars.huggingface.co/v1/production/uploads/65aab3940150f64adfcf5ba6/umkbppPt4grNdhLxP7ZUh.jpeg
Mohammad Omar Hadid
user
omarhadid
680733cf7722bb6407ca078c
Safwan AlModhayan
Kuwain 1.5B: An Arabic SLM via Language Injection
Enhancing existing models with new knowledge is a crucial aspect of AI
development. This paper introduces a novel method for integrating a new
language into a large language model (LLM). Our approach successfully
incorporates a previously unseen target language into an existing LLM without
compromising its prior knowledge. We trained a tiny model with 1.5 billion
parameters named Kuwain by injecting the Arabic language into a small
open-source model mainly trained in English. Our method demonstrates
significant improvements in Arabic language performance, with an average 8%
improvement across various benchmarks, while retaining the model's existing
knowledge with a minimum amount of the original model's data. This offers a
cost-effective alternative to training a comprehensive model in both English
and Arabic. The results highlight the potential for efficient, targeted
language model expansion without extensive retraining or resource-intensive
processes.
A method for integrating a new language into a large language model enhances performance in the target language while preserving existing knowledge.
large language model LLM language integration Kuwain Arabic parameter-efficient language model expansion