mLongT5: A Multilingual and Efficient Text-To-Text Transformer for
  Longer Sequences
We present our work on developing a multilingual, efficient text-to-text
transformer that is suitable for handling long inputs. This model, called
mLongT5, builds upon the architecture of LongT5, while leveraging the
multilingual datasets used for pretraining mT5 and the pretraining tasks of
UL2. We evaluate this model on a variety of multilingual summarization and
question-answering tasks, and the results show stronger performance for mLongT5
when compared to existing multilingual models such as mBART or M-BERT.
6466fef2cdea48159aa5c354
David Uthus
6466fef2cdea48159aa5c355
Santiago Ontañón
6466fef2cdea48159aa5c356
Joshua Ainslie
6466fef2cdea48159aa5c357
Mandy Guo
mLongT5: A Multilingual and Efficient Text-To-Text Transformer for
  Longer Sequences
We present our work on developing a multilingual, efficient text-to-text
transformer that is suitable for handling long inputs. This model, called
mLongT5, builds upon the architecture of LongT5, while leveraging the
multilingual datasets used for pretraining mT5 and the pretraining tasks of
UL2. We evaluate this model on a variety of multilingual summarization and
question-answering tasks, and the results show stronger performance for mLongT5
when compared to existing multilingual models such as mBART or M-BERT.
A multilingual text-to-text transformer called mLongT5, combining LongT5 with datasets from mT5 and UL2, outperforms existing models like mBART and M-BERT in summarization and question-answering tasks.
multilingual text-to-text transformer LongT5 mT5 UL2 summarization question-answering