Is Preference Alignment Always the Best Option to Enhance LLM-Based
  Translation? An Empirical Analysis
Neural metrics for machine translation (MT) evaluation have become
increasingly prominent due to their superior correlation with human judgments
compared to traditional lexical metrics. Researchers have therefore utilized
neural metrics through quality-informed decoding strategies, achieving better
results than likelihood-based methods. With the rise of Large Language Models
(LLMs), preference-based alignment techniques have gained attention for their
potential to enhance translation quality by optimizing model weights directly
on preferences induced by quality estimators. This study focuses on Contrastive
Preference Optimization (CPO) and conducts extensive experiments to evaluate
the impact of preference-based alignment on translation quality. Our findings
indicate that while CPO consistently outperforms Supervised Fine-Tuning (SFT)
on high-quality data with regard to the alignment metric, it may lead to
instability across downstream evaluation metrics, particularly between neural
and lexical ones. Additionally, we demonstrate that relying solely on the base
model for generating candidate translations achieves performance comparable to
using multiple external systems, while ensuring better consistency across
downstream metrics.
66fd33a20cde4879f9ab4654
Hippolyte Gisserot-Boukhlef
claimed_verified
2024-10-02T12:44:47.000Z
65fa95405355a52c784633fc
https://cdn-avatars.huggingface.co/v1/production/uploads/65fa95405355a52c784633fc/rSfBUHPa7eSAsLd8DuOq4.png
Hippolyte Gisserot-Boukhlef
user
hgissbkh
66fd33a20cde4879f9ab4655
Ricardo Rei
claimed_verified
2025-03-16T21:23:48.804Z
60d9a9791fa5d458da77754d
https://cdn-avatars.huggingface.co/v1/production/uploads/1624887555363-60d9a9791fa5d458da77754d.jpeg
Ricardo Rei
user
RicardoRei
66fd33a20cde4879f9ab4656
Emmanuel Malherbe
admin_assigned
2024-10-03T08:53:40.295Z
66f2d6a684a241caac8e16dc
/avatars/81acb87c2b07bea938251b40a2139911.svg
Emmanuel Malherbe
user
emmanuelmalherbe
66fd33a20cde4879f9ab4657
Céline Hudelot
admin_assigned
2024-10-03T08:53:47.051Z
61efea03a57920a251ec19b8
/avatars/f47c8e3cb17a2bf7d43f2c152bb86885.svg
Celine Hudelot
user
CelineH
66fd33a20cde4879f9ab4658
Pierre Colombo
admin_assigned
2024-10-03T08:53:53.386Z
644a900e3a619fe72b14af0f
/avatars/e2d5dac3d92757ed48e37e126a3464a3.svg
Colombo
user
PierreColombo
66fd33a20cde4879f9ab4659
Nuno M. Guerreiro
admin_assigned
2024-10-03T08:54:03.564Z
61f06c8ba57920a251ec1a5c
https://cdn-avatars.huggingface.co/v1/production/uploads/61f06c8ba57920a251ec1a5c/TmmSN9xWm1noLeAtsUSRk.jpeg
Nuno M. Guerreiro
user
nunonmg
Is Preference Alignment Always the Best Option to Enhance LLM-Based
  Translation? An Empirical Analysis
Neural metrics for machine translation (MT) evaluation have become
increasingly prominent due to their superior correlation with human judgments
compared to traditional lexical metrics. Researchers have therefore utilized
neural metrics through quality-informed decoding strategies, achieving better
results than likelihood-based methods. With the rise of Large Language Models
(LLMs), preference-based alignment techniques have gained attention for their
potential to enhance translation quality by optimizing model weights directly
on preferences induced by quality estimators. This study focuses on Contrastive
Preference Optimization (CPO) and conducts extensive experiments to evaluate
the impact of preference-based alignment on translation quality. Our findings
indicate that while CPO consistently outperforms Supervised Fine-Tuning (SFT)
on high-quality data with regard to the alignment metric, it may lead to
instability across downstream evaluation metrics, particularly between neural
and lexical ones. Additionally, we demonstrate that relying solely on the base
model for generating candidate translations achieves performance comparable to
using multiple external systems, while ensuring better consistency across
downstream metrics.
Contrastive Preference Optimization enhances translation quality on high-quality data but may cause instability across different evaluation metrics, while the base model's performance is comparable to using multiple external systems.
neural metrics machine translation quality-informed decoding likelihood-based methods Large Language Models preference-based alignment Contrastive Preference Optimization Supervised Fine-Tuning alignment metric lexical metrics