MI-Fuse: Label Fusion for Unsupervised Domain Adaptation with
  Closed-Source Large-Audio Language Model
Large audio-language models (LALMs) show strong zero-shot ability on speech
tasks, suggesting promise for speech emotion recognition (SER). However, SER in
real-world deployments often fails under domain mismatch, where source data are
unavailable and powerful LALMs are accessible only through an API. We ask:
given only unlabeled target-domain audio and an API-only LALM, can a student
model be adapted to outperform the LALM in the target domain? To this end, we
propose MI-Fuse, a denoised label fusion framework that supplements the LALM
with a source-domain trained SER classifier as an auxiliary teacher. The
framework draws multiple stochastic predictions from both teachers, weights
their mean distributions by mutual-information-based uncertainty, and
stabilizes training with an exponential moving average teacher. Experiments
across three public emotion datasets and six cross-domain transfers show
consistent gains, with the student surpassing the LALM and outperforming the
strongest baseline by 3.9%. This approach strengthens emotion-aware speech
systems without sharing source data, enabling realistic adaptation.
68d6276a8ccd91bdd39ffe5e
Hsiao-Ying Huang
claimed_verified
2025-09-26T12:08:56.676Z
6511c1101e3b49ac7932176e
/avatars/7719a3631cedbccf5285f606b4d3d9cf.svg
黃筱穎
user
MonicaHuang
68d6276a8ccd91bdd39ffe5f
Yi-Cheng Lin
claimed_verified
2025-09-26T12:08:59.674Z
650b0d66664f7b7d088ca281
/avatars/fce475c301f53e166fc3c8f5c5112c4a.svg
Yi-Cheng Lin
user
dlion168
68d6276a8ccd91bdd39ffe60
Hung-yi Lee
MI-Fuse: Label Fusion for Unsupervised Domain Adaptation with
  Closed-Source Large-Audio Language Model
Large audio-language models (LALMs) show strong zero-shot ability on speech
tasks, suggesting promise for speech emotion recognition (SER). However, SER in
real-world deployments often fails under domain mismatch, where source data are
unavailable and powerful LALMs are accessible only through an API. We ask:
given only unlabeled target-domain audio and an API-only LALM, can a student
model be adapted to outperform the LALM in the target domain? To this end, we
propose MI-Fuse, a denoised label fusion framework that supplements the LALM
with a source-domain trained SER classifier as an auxiliary teacher. The
framework draws multiple stochastic predictions from both teachers, weights
their mean distributions by mutual-information-based uncertainty, and
stabilizes training with an exponential moving average teacher. Experiments
across three public emotion datasets and six cross-domain transfers show
consistent gains, with the student surpassing the LALM and outperforming the
strongest baseline by 3.9%. This approach strengthens emotion-aware speech
systems without sharing source data, enabling realistic adaptation.
MI-Fuse, a denoised label fusion framework, enhances speech emotion recognition in target domains using an API-only LALM and a source-domain SER classifier, achieving better performance than the LALM and other baselines.
large audio-language models speech emotion recognition domain mismatch unlabeled target-domain audio denoised label fusion source-domain trained SER classifier mutual-information-based uncertainty exponential moving average teacher