KORMo: Korean Open Reasoning Model for Everyone
This work presents the first large-scale investigation into constructing a
fully open bilingual large language model (LLM) for a non-English language,
specifically Korean, trained predominantly on synthetic data. We introduce
KORMo-10B, a 10.8B-parameter model trained from scratch on a Korean-English
corpus in which 68.74% of the Korean portion is synthetic. Through systematic
experimentation, we demonstrate that synthetic data, when carefully curated
with balanced linguistic coverage and diverse instruction styles, does not
cause instability or degradation during large-scale pretraining. Furthermore,
the model achieves performance comparable to that of contemporary open-weight
multilingual baselines across a wide range of reasoning, knowledge, and
instruction-following benchmarks. Our experiments reveal two key findings: (1)
synthetic data can reliably sustain long-horizon pretraining without model
collapse, and (2) bilingual instruction tuning enables near-native reasoning
and discourse coherence in Korean. By fully releasing all components including
data, code, training recipes, and logs, this work establishes a transparent
framework for developing synthetic data-driven fully open models (FOMs) in
low-resource settings and sets a reproducible precedent for future multilingual
LLM research.
68ec6e86cd07fb414898c9af
Minjun Kim
claimed_verified
2025-10-13T10:05:34.054Z
6435a57b2d0ed796668d8a3f
https://cdn-avatars.huggingface.co/v1/production/uploads/6435a57b2d0ed796668d8a3f/fsOmfjqCS9TkI9NMjCFM2.png
min jun kim
user
mjkmain
68ec6e86cd07fb414898c9b0
Hyeonseok Lim
claimed_verified
2025-10-14T07:32:18.660Z
639dc5e07145123e0d496086
https://cdn-avatars.huggingface.co/v1/production/uploads/639dc5e07145123e0d496086/VKh779pf4dS7ZK9xEQtio.png
limhyeonseok
user
limhyeonseok
68ec6e86cd07fb414898c9b1
Hangyeol Yoo
claimed_verified
2025-10-13T10:05:31.900Z
64be114c12afb2f1194cf108
/avatars/ae00fde99b95fcc3a95aae8a558eb049.svg
Hangyeol Yoo
user
PerRing
68ec6e86cd07fb414898c9b2
Inho Won
68ec6e86cd07fb414898c9b3
Seungwoo Song
claimed_verified
2025-10-13T10:05:29.638Z
654c5e461fbef019f2ccbc3c
/avatars/c0527f73e36dd580b3ecb17efb11260c.svg
SeungWoo Song
user
sswoo123
68ec6e86cd07fb414898c9b4
Minkyung Cho
claimed_verified
2025-10-14T07:32:15.935Z
659f7345e98a198ba7fcb89e
/avatars/c40e8a111a3c1c08e7693156e3e2cbce.svg
Minkyung Cho
user
ada-flo
68ec6e86cd07fb414898c9b5
Junhun Yuk
68ec6e86cd07fb414898c9b6
Changsu Choi
68ec6e86cd07fb414898c9b7
Dongjae Shin
claimed_verified
2025-10-14T07:32:07.072Z
64a90711c05da19ca834f690
https://cdn-avatars.huggingface.co/v1/production/uploads/64a90711c05da19ca834f690/yS1Ja5poSjX2dxkBR4uWQ.jpeg
DongJae Shin
user
ShinDJ
68ec6e86cd07fb414898c9b8
Huige Lee
68ec6e86cd07fb414898c9b9
Hoyun Song
68ec6e86cd07fb414898c9ba
Alice Oh
68ec6e86cd07fb414898c9bb
Kyungtae Lim
KORMo: Korean Open Reasoning Model for Everyone
This work presents the first large-scale investigation into constructing a
fully open bilingual large language model (LLM) for a non-English language,
specifically Korean, trained predominantly on synthetic data. We introduce
KORMo-10B, a 10.8B-parameter model trained from scratch on a Korean-English
corpus in which 68.74% of the Korean portion is synthetic. Through systematic
experimentation, we demonstrate that synthetic data, when carefully curated
with balanced linguistic coverage and diverse instruction styles, does not
cause instability or degradation during large-scale pretraining. Furthermore,
the model achieves performance comparable to that of contemporary open-weight
multilingual baselines across a wide range of reasoning, knowledge, and
instruction-following benchmarks. Our experiments reveal two key findings: (1)
synthetic data can reliably sustain long-horizon pretraining without model
collapse, and (2) bilingual instruction tuning enables near-native reasoning
and discourse coherence in Korean. By fully releasing all components including
data, code, training recipes, and logs, this work establishes a transparent
framework for developing synthetic data-driven fully open models (FOMs) in
low-resource settings and sets a reproducible precedent for future multilingual
LLM research.
A large-scale investigation into constructing a fully open bilingual LLM for Korean using synthetic data demonstrates that such data can sustain pretraining and achieve performance comparable to multilingual baselines.
large language model LLM KORMo-10B synthetic data Korean-English corpus pretraining model collapse bilingual instruction tuning fully open models FOMs low-resource settings