REINA: Regularized Entropy Information-Based Loss for Efficient
  Simultaneous Speech Translation
Simultaneous Speech Translation (SimulST) systems stream in audio while
simultaneously emitting translated text or speech. Such systems face the
significant challenge of balancing translation quality and latency. We
introduce a strategy to optimize this tradeoff: wait for more input only if you
gain information by doing so. Based on this strategy, we present Regularized
Entropy INformation Adaptation (REINA), a novel loss to train an adaptive
policy using an existing non-streaming translation model. We derive REINA from
information theory principles and show that REINA helps push the reported
Pareto frontier of the latency/quality tradeoff over prior works. Utilizing
REINA, we train a SimulST model on French, Spanish and German, both from and
into English. Training on only open source or synthetically generated data, we
achieve state-of-the-art (SOTA) streaming results for models of comparable
size. We also introduce a metric for streaming efficiency, quantitatively
showing REINA improves the latency/quality trade-off by as much as 21% compared
to prior approaches, normalized against non-streaming baseline BLEU scores.
68958b7348b0ae5ca2710e0e
Nameer Hirschkind
68958b7348b0ae5ca2710e0f
Joseph Liu
68958b7348b0ae5ca2710e10
Mahesh Kumar Nandwana
claimed_verified
2025-08-11T06:50:17.863Z
667f437aa94430cd2e723346
/avatars/c7d71f6789756082e129b0b9cf9dcfda.svg
Mahesh Nandwana
user
mnandwana
68958b7348b0ae5ca2710e11
Xiao Yu
REINA: Regularized Entropy Information-Based Loss for Efficient
  Simultaneous Speech Translation
Simultaneous Speech Translation (SimulST) systems stream in audio while
simultaneously emitting translated text or speech. Such systems face the
significant challenge of balancing translation quality and latency. We
introduce a strategy to optimize this tradeoff: wait for more input only if you
gain information by doing so. Based on this strategy, we present Regularized
Entropy INformation Adaptation (REINA), a novel loss to train an adaptive
policy using an existing non-streaming translation model. We derive REINA from
information theory principles and show that REINA helps push the reported
Pareto frontier of the latency/quality tradeoff over prior works. Utilizing
REINA, we train a SimulST model on French, Spanish and German, both from and
into English. Training on only open source or synthetically generated data, we
achieve state-of-the-art (SOTA) streaming results for models of comparable
size. We also introduce a metric for streaming efficiency, quantitatively
showing REINA improves the latency/quality trade-off by as much as 21% compared
to prior approaches, normalized against non-streaming baseline BLEU scores.
A novel loss function, REINA, optimizes the latency-quality tradeoff in Simultaneous Speech Translation by adaptively waiting for more input based on information gain.
Simultaneous Speech Translation SimulST Regularized Entropy INformation Adaptation REINA information theory Pareto frontier streaming efficiency BLEU scores