FuxiMT: Sparsifying Large Language Models for Chinese-Centric
  Multilingual Machine Translation
In this paper, we present FuxiMT, a novel Chinese-centric multilingual
machine translation model powered by a sparsified large language model (LLM).
We adopt a two-stage strategy to train FuxiMT. We first pre-train the model on
a massive Chinese corpus and then conduct multilingual fine-tuning on a large
parallel dataset encompassing 65 languages. FuxiMT incorporates
Mixture-of-Experts (MoEs) and employs a curriculum learning strategy for robust
performance across various resource levels. Experimental results demonstrate
that FuxiMT significantly outperforms strong baselines, including
state-of-the-art LLMs and machine translation models, particularly under
low-resource scenarios. Furthermore, FuxiMT exhibits remarkable zero-shot
translation capabilities for unseen language pairs, indicating its potential to
bridge communication gaps where parallel data are scarce or unavailable.
683330f7e10e89e250807ec2
Shaolin Zhu
683330f7e10e89e250807ec3
Tianyu Dong
683330f7e10e89e250807ec4
Bo Li
claimed_verified
2025-05-26T08:10:37.787Z
6582c482f3006507ea10302a
https://cdn-avatars.huggingface.co/v1/production/uploads/6582c482f3006507ea10302a/KbgSsq0FnbMngBcWPhIXi.jpeg
Bo Li
user
liboaccn
683330f7e10e89e250807ec5
Deyi Xiong
FuxiMT: Sparsifying Large Language Models for Chinese-Centric
  Multilingual Machine Translation
In this paper, we present FuxiMT, a novel Chinese-centric multilingual
machine translation model powered by a sparsified large language model (LLM).
We adopt a two-stage strategy to train FuxiMT. We first pre-train the model on
a massive Chinese corpus and then conduct multilingual fine-tuning on a large
parallel dataset encompassing 65 languages. FuxiMT incorporates
Mixture-of-Experts (MoEs) and employs a curriculum learning strategy for robust
performance across various resource levels. Experimental results demonstrate
that FuxiMT significantly outperforms strong baselines, including
state-of-the-art LLMs and machine translation models, particularly under
low-resource scenarios. Furthermore, FuxiMT exhibits remarkable zero-shot
translation capabilities for unseen language pairs, indicating its potential to
bridge communication gaps where parallel data are scarce or unavailable.
FuxiMT, a Chinese-centric multilingual machine translation model utilizing a sparsified large language model, demonstrates superior performance in low-resource scenarios and strong zero-shot capabilities across 65 languages.
sparsified large language model Mixture-of-Experts (MoEs) curriculum learning zero-shot translation