BigTrans: Augmenting Large Language Models with Multilingual Translation
  Capability over 100 Languages
Large language models (LLMs) demonstrate promising translation performance
among various natural languages. However, many LLMs especially the open-sourced
ones, such as BLOOM and LLaMA, are English-dominant and support only dozens of
natural languages, making the potential of LLMs on language translation less
explored. In this work, we present BigTrans which adapts LLaMA that covers only
20 languages and enhances it with multilingual translation capability on more
than 100 languages. BigTrans is built upon LLaMA-13B and it is optimized in
three steps. First, we continue training LLaMA with massive Chinese monolingual
data. Second, we continue training the model with a large-scale parallel
dataset that covers 102 natural languages. Third, we instruct-tune the
foundation model with multilingual translation instructions, leading to our
BigTrans model. The preliminary experiments on multilingual translation show
that BigTrans performs comparably with ChatGPT and Google Translate in many
languages and even outperforms ChatGPT in 8 language pairs. We release the
BigTrans model and hope it can advance the research progress.
64756221b68461d5cf7ec0e5
Wen Yang
claimed_verified
2023-05-30T14:15:21.074Z
641fd72a73cfc036ddbf69c8
https://cdn-avatars.huggingface.co/v1/production/uploads/641fd72a73cfc036ddbf69c8/W4HG5HRm-OkzVpOotgu3m.jpeg
WenYang
user
James-WYang
64756221b68461d5cf7ec0e6
Chong Li
64756221b68461d5cf7ec0e7
Jiajun Zhang
64756221b68461d5cf7ec0e8
Chengqing Zong
BigTrans: Augmenting Large Language Models with Multilingual Translation
  Capability over 100 Languages
Large language models (LLMs) demonstrate promising translation performance
among various natural languages. However, many LLMs especially the open-sourced
ones, such as BLOOM and LLaMA, are English-dominant and support only dozens of
natural languages, making the potential of LLMs on language translation less
explored. In this work, we present BigTrans which adapts LLaMA that covers only
20 languages and enhances it with multilingual translation capability on more
than 100 languages. BigTrans is built upon LLaMA-13B and it is optimized in
three steps. First, we continue training LLaMA with massive Chinese monolingual
data. Second, we continue training the model with a large-scale parallel
dataset that covers 102 natural languages. Third, we instruct-tune the
foundation model with multilingual translation instructions, leading to our
BigTrans model. The preliminary experiments on multilingual translation show
that BigTrans performs comparably with ChatGPT and Google Translate in many
languages and even outperforms ChatGPT in 8 language pairs. We release the
BigTrans model and hope it can advance the research progress.
BigTrans, an enhanced version of LLaMA, achieves competitive multilingual translation performance across over 100 languages.
Large language models English-dominant BigTrans LLaMA Chinese monolingual data parallel dataset multilingual translation instruct-tuning ChatGPT Google Translate