Low-Resource Machine Translation through the Lens of Personalized
  Federated Learning
We present a new approach based on the Personalized Federated Learning
algorithm MeritFed that can be applied to Natural Language Tasks with
heterogeneous data. We evaluate it on the Low-Resource Machine Translation
task, using the dataset from the Large-Scale Multilingual Machine Translation
Shared Task (Small Track #2) and the subset of Sami languages from the
multilingual benchmark for Finno-Ugric languages. In addition to its
effectiveness, MeritFed is also highly interpretable, as it can be applied to
track the impact of each language used for training. Our analysis reveals that
target dataset size affects weight distribution across auxiliary languages,
that unrelated languages do not interfere with the training, and auxiliary
optimizer parameters have minimal impact. Our approach is easy to apply with a
few lines of code, and we provide scripts for reproducing the experiments at
https://github.com/VityaVitalich/MeritFed
6679316cae4944f2cca7d1bf
Viktor Moskvoretskii
claimed_verified
2024-06-24T13:10:03.026Z
63bbfd74141c7d395c471768
https://cdn-avatars.huggingface.co/v1/production/uploads/1673264437106-noauth.jpeg
Viktor Moskvoretskii
user
VityaVitalich
6679316cae4944f2cca7d1c0
Nazarii Tupitsa
6679316cae4944f2cca7d1c1
Chris Biemann
6679316cae4944f2cca7d1c2
Samuel Horváth
6679316cae4944f2cca7d1c3
Eduard Gorbunov
6679316cae4944f2cca7d1c4
Irina Nikishina
admin_assigned
2024-06-24T14:21:11.465Z
6458a9ed6fa580137af4c710
/avatars/8b9f99537adf30fee6aa7ae61ab7235b.svg
Irina Nikishina
user
lilaspourpre
Low-Resource Machine Translation through the Lens of Personalized
  Federated Learning
We present a new approach based on the Personalized Federated Learning
algorithm MeritFed that can be applied to Natural Language Tasks with
heterogeneous data. We evaluate it on the Low-Resource Machine Translation
task, using the dataset from the Large-Scale Multilingual Machine Translation
Shared Task (Small Track #2) and the subset of Sami languages from the
multilingual benchmark for Finno-Ugric languages. In addition to its
effectiveness, MeritFed is also highly interpretable, as it can be applied to
track the impact of each language used for training. Our analysis reveals that
target dataset size affects weight distribution across auxiliary languages,
that unrelated languages do not interfere with the training, and auxiliary
optimizer parameters have minimal impact. Our approach is easy to apply with a
few lines of code, and we provide scripts for reproducing the experiments at
https://github.com/VityaVitalich/MeritFed
MeritFed, a personalized federated learning algorithm, effectively handles low-resource machine translation with heterogeneous data and is interpretable in tracking language impacts.
Personalized Federated Learning MeritFed Federated Learning Low-Resource Machine Translation Large-Scale Multilingual Machine Translation auxiliary languages auxiliary optimizer parameters