Advancing Speech Understanding in Speech-Aware Language Models with GRPO
In this paper, we introduce a Group Relative Policy Optimization (GRPO)-based
method for training Speech-Aware Large Language Models (SALLMs) on open-format
speech understanding tasks, such as Spoken Question Answering and Automatic
Speech Translation. SALLMs have proven highly effective for speech
understanding tasks. GRPO has recently gained traction for its efficiency in
training LLMs, and prior work has explored its application to SALLMs, primarily
in multiple-choice tasks. Building on this, we focus on open-format tasks that
better reflect the generative abilities of the models. Our approach leverages
GRPO with BLEU as the reward signal to optimize SALLMs, and we demonstrate
empirically that it surpasses standard SFT across several key metrics. Finally,
we explore the potential of incorporating off-policy samples within GRPO for
these tasks, highlighting avenues for further improvement and further research.
68d4e0e836950a9dff156933
Avishai Elmakies
claimed_verified
2025-09-25T07:10:44.431Z
644662145004f2cb3af08b27
/avatars/5f2af24c7410a5db46374d0b84fb479d.svg
Avishai Elmakies
user
avishai-elmakies
68d4e0e836950a9dff156934
Hagai Aronowitz
admin_assigned
2025-09-25T11:36:38.678Z
643425b4a4c9c55871a7a02b
/avatars/eb2f357888159f5120bbf70a40cb089d.svg
Hagai Aronowitz
user
hagaia
68d4e0e836950a9dff156935
Nimrod Shabtay
claimed_verified
2025-09-25T07:10:41.442Z
62bedff7304b82a773bf8c1b
/avatars/f9e79dc196caa95c220127c6212e9944.svg
Nimrod Shabtay
user
NimrodShabtay1986
68d4e0e836950a9dff156936
Eli Schwartz
68d4e0e836950a9dff156937
Ron Hoory
admin_assigned
2025-09-25T11:36:49.661Z
68b3fe80a9f319567d039ec0
/avatars/672f169213139d21c7767021a0110402.svg
Ron Hoory
user
rhoory
68d4e0e836950a9dff156938
Avihu Dekel
admin_assigned
2025-09-25T11:36:56.726Z
63b7e09359060ca9f4c4de35
https://cdn-avatars.huggingface.co/v1/production/uploads/63b7e09359060ca9f4c4de35/0PH1dWNfcXTQ9H0QAeD91.jpeg
Avihu Dekel
user
Avihu
Advancing Speech Understanding in Speech-Aware Language Models with GRPO
In this paper, we introduce a Group Relative Policy Optimization (GRPO)-based
method for training Speech-Aware Large Language Models (SALLMs) on open-format
speech understanding tasks, such as Spoken Question Answering and Automatic
Speech Translation. SALLMs have proven highly effective for speech
understanding tasks. GRPO has recently gained traction for its efficiency in
training LLMs, and prior work has explored its application to SALLMs, primarily
in multiple-choice tasks. Building on this, we focus on open-format tasks that
better reflect the generative abilities of the models. Our approach leverages
GRPO with BLEU as the reward signal to optimize SALLMs, and we demonstrate
empirically that it surpasses standard SFT across several key metrics. Finally,
we explore the potential of incorporating off-policy samples within GRPO for
these tasks, highlighting avenues for further improvement and further research.
A Group Relative Policy Optimization (GRPO)-based method using BLEU as a reward signal outperforms standard SFT for open-format speech understanding tasks like Spoken Question Answering and Automatic Speech Translation.
Group Relative Policy Optimization GRPO Speech-Aware Large Language Models SALLMs Spoken Question Answering Automatic Speech Translation BLEU standard SFT off-policy samples