MultiTACRED: A Multilingual Version of the TAC Relation Extraction Dataset
Relation extraction (RE) is a fundamental task in information extraction,
whose extension to multilingual settings has been hindered by the lack of
supervised resources comparable in size to large English datasets such as
TACRED (Zhang et al., 2017). To address this gap, we introduce the MultiTACRED
dataset, covering 12 typologically diverse languages from 9 language families,
which is created by machine-translating TACRED instances and automatically
projecting their entity annotations. We analyze translation and annotation
projection quality, identify error categories, and experimentally evaluate
fine-tuned pretrained mono- and multilingual language models in common transfer
learning scenarios. Our analyses show that machine translation is a viable
strategy to transfer RE instances, with native speakers judging more than 83%
of the translated instances to be linguistically and semantically acceptable.
We find monolingual RE model performance to be comparable to the English
original for many of the target languages, and that multilingual models trained
on a combination of English and target language data can outperform their
monolingual counterparts. However, we also observe a variety of translation and
annotation projection errors, both due to the MT systems and linguistic
features of the target languages, such as pronoun-dropping, compounding and
inflection, that degrade dataset quality and RE model performance.