Do You Hear What I Mean? Quantifying the Instruction-Perception Gap in
  Instruction-Guided Expressive Text-To-Speech Systems
Instruction-guided text-to-speech (ITTS) enables users to control speech
generation through natural language prompts, offering a more intuitive
interface than traditional TTS. However, the alignment between user style
instructions and listener perception remains largely unexplored. This work
first presents a perceptual analysis of ITTS controllability across two
expressive dimensions (adverbs of degree and graded emotion intensity) and
collects human ratings on speaker age and word-level emphasis attributes. To
comprehensively reveal the instruction-perception gap, we provide a data
collection with large-scale human evaluations, named Expressive VOice Control
(E-VOC) corpus. Furthermore, we reveal that (1) gpt-4o-mini-tts is the most
reliable ITTS model with great alignment between instruction and generated
utterances across acoustic dimensions. (2) The 5 analyzed ITTS systems tend to
generate Adult voices even when the instructions ask to use child or Elderly
voices. (3) Fine-grained control remains a major challenge, indicating that
most ITTS systems have substantial room for improvement in interpreting
slightly different attribute instructions.
68d0e3b48adc5cd018d15b81
Yi-Cheng Lin
claimed_verified
2025-09-26T12:23:33.792Z
650b0d66664f7b7d088ca281
/avatars/fce475c301f53e166fc3c8f5c5112c4a.svg
Yi-Cheng Lin
user
dlion168
68d0e3b48adc5cd018d15b82
Huang-Cheng Chou
68d0e3b48adc5cd018d15b83
Tzu-Chieh Wei
68d0e3b48adc5cd018d15b84
Kuan-Yu Chen
68d0e3b48adc5cd018d15b85
Hung-yi Lee
Do You Hear What I Mean? Quantifying the Instruction-Perception Gap in
  Instruction-Guided Expressive Text-To-Speech Systems
Instruction-guided text-to-speech (ITTS) enables users to control speech
generation through natural language prompts, offering a more intuitive
interface than traditional TTS. However, the alignment between user style
instructions and listener perception remains largely unexplored. This work
first presents a perceptual analysis of ITTS controllability across two
expressive dimensions (adverbs of degree and graded emotion intensity) and
collects human ratings on speaker age and word-level emphasis attributes. To
comprehensively reveal the instruction-perception gap, we provide a data
collection with large-scale human evaluations, named Expressive VOice Control
(E-VOC) corpus. Furthermore, we reveal that (1) gpt-4o-mini-tts is the most
reliable ITTS model with great alignment between instruction and generated
utterances across acoustic dimensions. (2) The 5 analyzed ITTS systems tend to
generate Adult voices even when the instructions ask to use child or Elderly
voices. (3) Fine-grained control remains a major challenge, indicating that
most ITTS systems have substantial room for improvement in interpreting
slightly different attribute instructions.
Research on ITTS reveals gaps between user instructions and listener perception, highlighting challenges in fine-grained control and voice attribute interpretation.
instruction-guided text-to-speech ITTS perceptual analysis expressive dimensions adverbs of degree graded emotion intensity speaker age word-level emphasis Expressive VOice Control E-VOC corpus gpt-4o-mini-tts acoustic dimensions fine-grained control