When Life Gives You Samples: The Benefits of Scaling up Inference
  Compute for Multilingual LLMs
Recent advancements in large language models (LLMs) have shifted focus toward
scaling inference-time compute, improving performance without retraining the
model. A common approach is to sample multiple outputs in parallel, and select
one of these as the final output. However, work to date has focused on English
and a handful of domains such as math and code. In contrast, we are most
interested in techniques that generalize across open-ended tasks, formally
verifiable tasks, and across languages. In this work, we study how to robustly
scale inference-time compute for open-ended generative tasks in a multilingual,
multi-task setting.
  Our findings show that both sampling strategy based on temperature variation
and selection strategy must be adapted to account for diverse domains and
varied language settings. We evaluate existing selection methods, revealing
that strategies effective in English often fail to generalize across languages.
We propose novel sampling and selection strategies specifically adapted for
multilingual and multi-task inference scenarios, and show they yield notable
gains across languages and tasks. In particular, our combined sampling and
selection methods lead to an average +6.8 jump in win-rates for our 8B models
on m-ArenaHard-v2.0 prompts, against proprietary models such as Gemini. At
larger scale, Command-A (111B model) equipped with our methods, shows +9.0
improvement in win-rates on the same benchmark with just five samples against
single-sample decoding, a substantial increase at minimal cost. Our results
underscore the need for language- and task-aware approaches to inference-time
compute, aiming to democratize performance improvements in underrepresented
languages.
685cdd71696820ba1f28f2b8
Ammar Khairi
claimed_verified
2025-06-26T09:23:26.999Z
677cfa6cac2db4c2265edb26
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/Kbi96ndfY-CIuJNd2TRZt.jpeg
Ammar Khairi
user
ammar-cohere
685cdd71696820ba1f28f2b9
Daniel D'souza
claimed_verified
2025-07-10T07:10:21.421Z
6658011eaba105a066e37e1b
https://cdn-avatars.huggingface.co/v1/production/uploads/6658011eaba105a066e37e1b/VPwyTv1bnVMQbVMoMQzcf.jpeg
Daniel D'souza
user
dsouzadaniel
685cdd71696820ba1f28f2ba
Ye Shen
685cdd71696820ba1f28f2bb
Julia Kreutzer
685cdd71696820ba1f28f2bc
Sara Hooker
When Life Gives You Samples: The Benefits of Scaling up Inference
  Compute for Multilingual LLMs
Recent advancements in large language models (LLMs) have shifted focus toward
scaling inference-time compute, improving performance without retraining the
model. A common approach is to sample multiple outputs in parallel, and select
one of these as the final output. However, work to date has focused on English
and a handful of domains such as math and code. In contrast, we are most
interested in techniques that generalize across open-ended tasks, formally
verifiable tasks, and across languages. In this work, we study how to robustly
scale inference-time compute for open-ended generative tasks in a multilingual,
multi-task setting.
  Our findings show that both sampling strategy based on temperature variation
and selection strategy must be adapted to account for diverse domains and
varied language settings. We evaluate existing selection methods, revealing
that strategies effective in English often fail to generalize across languages.
We propose novel sampling and selection strategies specifically adapted for
multilingual and multi-task inference scenarios, and show they yield notable
gains across languages and tasks. In particular, our combined sampling and
selection methods lead to an average +6.8 jump in win-rates for our 8B models
on m-ArenaHard-v2.0 prompts, against proprietary models such as Gemini. At
larger scale, Command-A (111B model) equipped with our methods, shows +9.0
improvement in win-rates on the same benchmark with just five samples against
single-sample decoding, a substantial increase at minimal cost. Our results
underscore the need for language- and task-aware approaches to inference-time
compute, aiming to democratize performance improvements in underrepresented
languages.
The study examines and proposes new sampling and selection strategies to enhance inference-time compute for multilingual and multi-task large language models, demonstrating significant improvements in win-rates across various languages and tasks.
sampling strategy selection strategy temperature variation open-ended generative tasks multilingual multi-task m-ArenaHard-v2.0 win-rates Command-A single-sample decoding language-aware task-aware