LLaMAX2: Your Translation-Enhanced Model also Performs Well in Reasoning
General Large Language Models (LLMs) excel in reasoning, but those enhanced
for translation struggle with reasoning tasks. To address this, we propose a
novel translationenhanced recipe that begins with instruct models and applies
layer-selective tuning only on parallel data. Following this pipeline, we
introduce the Qwen3-XPlus models, which demonstrate significant improvements in
translation performance across both high- and lowresource languages, achieving
15+ spBLEU and 40+ xComet in low-resource languages, like Swahili.
Interestingly, training only with small parallel datasets, Qwen3-XPlus achieves
an average improvement of 1+ points on 7 multilingual tasks while maintaining
proficiency comparable to the Qwen3 instruct model in 15 popular reasoning
datasets. This work offers a promising approach to multilingual enhancement,
significantly reducing complexity and enhancing accessibility for a wider range
of languages. The code and model are publicly available.
68ec701fcd07fb414898c9d2
Changjiang Gao
68ec701fcd07fb414898c9d3
Zixian Huang
68ec701fcd07fb414898c9d4
Jingyang Gong
68ec701fcd07fb414898c9d5
Shujian Huang
68ec701fcd07fb414898c9d6
Lei Li
68ec701fcd07fb414898c9d7
Fei Yuan
LLaMAX2: Your Translation-Enhanced Model also Performs Well in Reasoning
General Large Language Models (LLMs) excel in reasoning, but those enhanced
for translation struggle with reasoning tasks. To address this, we propose a
novel translationenhanced recipe that begins with instruct models and applies
layer-selective tuning only on parallel data. Following this pipeline, we
introduce the Qwen3-XPlus models, which demonstrate significant improvements in
translation performance across both high- and lowresource languages, achieving
15+ spBLEU and 40+ xComet in low-resource languages, like Swahili.
Interestingly, training only with small parallel datasets, Qwen3-XPlus achieves
an average improvement of 1+ points on 7 multilingual tasks while maintaining
proficiency comparable to the Qwen3 instruct model in 15 popular reasoning
datasets. This work offers a promising approach to multilingual enhancement,
significantly reducing complexity and enhancing accessibility for a wider range
of languages. The code and model are publicly available.
A novel translation-enhanced recipe using layer-selective tuning on parallel data improves translation performance in both high- and low-resource languages while maintaining reasoning proficiency.
instruct models layer-selective tuning parallel data Qwen3-XPlus models spBLEU xComet multilingual tasks reasoning datasets