Kanana: Compute-efficient Bilingual Language Models
We introduce Kanana, a series of bilingual language models that demonstrate
exceeding performance in Korean and competitive performance in English. The
computational cost of Kanana is significantly lower than that of
state-of-the-art models of similar size. The report details the techniques
employed during pre-training to achieve compute-efficient yet competitive
models, including high quality data filtering, staged pre-training, depth
up-scaling, and pruning and distillation. Furthermore, the report outlines the
methodologies utilized during the post-training of the Kanana models,
encompassing supervised fine-tuning and preference optimization, aimed at
enhancing their capability for seamless interaction with users. Lastly, the
report elaborates on plausible approaches used for language model adaptation to
specific scenarios, such as embedding, retrieval augmented generation, and
function calling. The Kanana model series spans from 2.1B to 32.5B parameters
with 2.1B models (base, instruct, embedding) publicly released to promote
research on Korean language models.
67bfe1bf4426925c82fe5953
Kanana LLM Team
67bfe1bf4426925c82fe5954
Yunju Bak
admin_assigned
2025-02-27T12:55:35.505Z
64d08bd75de9e1e911b24226
/avatars/e572bb47659393573a0c1fb3d333dd7b.svg
Yunju Bak
user
yunjubak63
67bfe1bf4426925c82fe5955
Hojin Lee
67bfe1bf4426925c82fe5956
Minho Ryu
claimed_verified
2025-02-27T09:14:17.979Z
60436d159e905013ae8715d7
https://cdn-avatars.huggingface.co/v1/production/uploads/1623809612769-60436d159e905013ae8715d7.jpeg
Minho Ryu
user
bzantium
67bfe1bf4426925c82fe5957
Jiyeon Ham
claimed_verified
2025-02-27T09:14:11.786Z
66ebb4fdc5b2c25450fd17de
/avatars/e6b40dcbe2eba838ba21be9221758a3c.svg
Jiyeon Ham
user
jiyeonham
67bfe1bf4426925c82fe5958
Seungjae Jung
67bfe1bf4426925c82fe5959
Daniel Wontae Nam
claimed_verified
2025-02-27T09:14:09.613Z
66c82a50c1b3c03c61aea140
/avatars/3c508f96bdca2f2ce9746d3decd4718e.svg
daniel nam
user
daniel-rl2
67bfe1bf4426925c82fe595a
Taegyeong Eo
67bfe1bf4426925c82fe595b
Donghun Lee
67bfe1bf4426925c82fe595c
Doohae Jung
claimed_verified
2025-02-27T09:14:06.858Z
6142e17fe9e656d4459121e4
/avatars/6baebd4598a845ec7fdb735eb0d53139.svg
Doohae Jung
user
Doohae
67bfe1bf4426925c82fe595d
Boseop Kim
claimed_verified
2025-02-27T09:14:01.989Z
60f559be68ee3ef098e407cf
/avatars/e1f00ff1c1c9fa7f591535d39c7d5e44.svg
Boseop Kim
user
seopbo
67bfe1bf4426925c82fe595e
Nayeon Kim
claimed_verified
2025-02-27T09:14:13.867Z
6605028007a154c768e1c4c7
/avatars/88678edb83fdb466067e38acd22d07de.svg
Nayeon Kim
user
lana-ny
67bfe1bf4426925c82fe595f
Jaesun Park
claimed_verified
2025-02-27T09:14:15.898Z
6136f65440e43b8f748a0833
/avatars/f72a5ae3d3e94485de8aed8df94abdad.svg
Jaesun Park
user
jaesun
67bfe1bf4426925c82fe5960
Hyunho Kim
67bfe1bf4426925c82fe5961
Hyunwoong Ko
admin_assigned
2025-02-27T12:58:05.546Z
5fd888cf61e46993190ce543
https://cdn-avatars.huggingface.co/v1/production/uploads/1634604273263-5fd888cf61e46993190ce543.jpeg
Hyunwoong Ko
user
hyunwoongko
67bfe1bf4426925c82fe5962
Changmin Lee
claimed_verified
2025-02-27T09:14:04.506Z
63d268bb57ab367124ea7b75
/avatars/11312cde1e9f077aa9e5103b48be5de6.svg
Changmin Lee
user
changminlee
67bfe1bf4426925c82fe5963
Kyoung-Woon On
admin_assigned
2025-02-27T12:58:11.269Z
62bd31e1d2c8a6542f53fcba
/avatars/4ac18a7bcaf9dd3885b0478dea90818f.svg
Kyoung-Woon On
user
kloud
67bfe1bf4426925c82fe5964
Seulye Baeg
67bfe1bf4426925c82fe5965
Junrae Cho
67bfe1bf4426925c82fe5966
Sunghee Jung
claimed_verified
2025-04-03T08:29:34.214Z
65e30342e8b017ee1384824c
/avatars/e5d07b037f611ccfaf719959d971d102.svg
Sunghee Jung
user
hash2430
67bfe1bf4426925c82fe5967
Jieun Kang
67bfe1bf4426925c82fe5968
EungGyun Kim
67bfe1bf4426925c82fe5969
Eunhwa Kim
67bfe1bf4426925c82fe596a
Byeongil Ko
67bfe1bf4426925c82fe596b
Daniel Lee
67bfe1bf4426925c82fe596c
Minchul Lee
67bfe1bf4426925c82fe596d
Miok Lee
67bfe1bf4426925c82fe596e
Shinbok Lee
67bfe1bf4426925c82fe596f
Gaeun Seo
admin_assigned
2025-02-27T12:59:39.670Z
63148a8f5f47a18962765802
/avatars/bc58a863727794006dddf758efa09411.svg
gaeunseo
user
gaeunseo
Kanana: Compute-efficient Bilingual Language Models
We introduce Kanana, a series of bilingual language models that demonstrate
exceeding performance in Korean and competitive performance in English. The
computational cost of Kanana is significantly lower than that of
state-of-the-art models of similar size. The report details the techniques
employed during pre-training to achieve compute-efficient yet competitive
models, including high quality data filtering, staged pre-training, depth
up-scaling, and pruning and distillation. Furthermore, the report outlines the
methodologies utilized during the post-training of the Kanana models,
encompassing supervised fine-tuning and preference optimization, aimed at
enhancing their capability for seamless interaction with users. Lastly, the
report elaborates on plausible approaches used for language model adaptation to
specific scenarios, such as embedding, retrieval augmented generation, and
function calling. The Kanana model series spans from 2.1B to 32.5B parameters
with 2.1B models (base, instruct, embedding) publicly released to promote
research on Korean language models.
Kanana, a series of bilingual language models, achieves superior performance in Korean and competitive performance in English with lower computational costs through efficient pre-training and post-training techniques.
high quality data filtering staged pre-training depth up-scaling pruning distillation supervised fine-tuning preference optimization embedding retrieval augmented generation function calling