AudioStory: Generating Long-Form Narrative Audio with Large Language
  Models
Recent advances in text-to-audio (TTA) generation excel at synthesizing short
audio clips but struggle with long-form narrative audio, which requires
temporal coherence and compositional reasoning. To address this gap, we propose
AudioStory, a unified framework that integrates large language models (LLMs)
with TTA systems to generate structured, long-form audio narratives. AudioStory
possesses strong instruction-following reasoning generation capabilities. It
employs LLMs to decompose complex narrative queries into temporally ordered
sub-tasks with contextual cues, enabling coherent scene transitions and
emotional tone consistency. AudioStory has two appealing features: (1)
Decoupled bridging mechanism: AudioStory disentangles LLM-diffuser
collaboration into two specialized components, i.e., a bridging query for
intra-event semantic alignment and a residual query for cross-event coherence
preservation. (2) End-to-end training: By unifying instruction comprehension
and audio generation within a single end-to-end framework, AudioStory
eliminates the need for modular training pipelines while enhancing synergy
between components. Furthermore, we establish a benchmark AudioStory-10K,
encompassing diverse domains such as animated soundscapes and natural sound
narratives. Extensive experiments show the superiority of AudioStory on both
single-audio generation and narrative audio generation, surpassing prior TTA
baselines in both instruction-following ability and audio fidelity. Our code is
available at https://github.com/TencentARC/AudioStory
68afd25f245176306494cc68
Yuxin Guo
68afd25f245176306494cc69
Teng Wang
claimed_verified
2025-08-28T08:49:54.138Z
63a9414e32ed73936ec0a0c8
/avatars/f181e1bb480502c2680be5296a036bdd.svg
wybertwang
user
wybertwang
68afd25f245176306494cc6a
Yuying Ge
68afd25f245176306494cc6b
Shijie Ma
claimed_verified
2025-08-28T18:17:05.402Z
67d30d9ae45dc43004b31425
/avatars/714f4c88710ab4cc75dda49c76dd3b09.svg
Shijie Ma
user
msj9817
68afd25f245176306494cc6c
Yixiao Ge
68afd25f245176306494cc6d
Wei Zou
68afd25f245176306494cc6e
Ying Shan
AudioStory: Generating Long-Form Narrative Audio with Large Language
  Models
Recent advances in text-to-audio (TTA) generation excel at synthesizing short
audio clips but struggle with long-form narrative audio, which requires
temporal coherence and compositional reasoning. To address this gap, we propose
AudioStory, a unified framework that integrates large language models (LLMs)
with TTA systems to generate structured, long-form audio narratives. AudioStory
possesses strong instruction-following reasoning generation capabilities. It
employs LLMs to decompose complex narrative queries into temporally ordered
sub-tasks with contextual cues, enabling coherent scene transitions and
emotional tone consistency. AudioStory has two appealing features: (1)
Decoupled bridging mechanism: AudioStory disentangles LLM-diffuser
collaboration into two specialized components, i.e., a bridging query for
intra-event semantic alignment and a residual query for cross-event coherence
preservation. (2) End-to-end training: By unifying instruction comprehension
and audio generation within a single end-to-end framework, AudioStory
eliminates the need for modular training pipelines while enhancing synergy
between components. Furthermore, we establish a benchmark AudioStory-10K,
encompassing diverse domains such as animated soundscapes and natural sound
narratives. Extensive experiments show the superiority of AudioStory on both
single-audio generation and narrative audio generation, surpassing prior TTA
baselines in both instruction-following ability and audio fidelity. Our code is
available at https://github.com/TencentARC/AudioStory
AudioStory integrates large language models with text-to-audio systems to generate coherent, long-form audio narratives through a unified framework with decoupled bridging mechanisms and end-to-end training.
large language models text-to-audio AudioStory instruction-following reasoning temporally ordered sub-tasks contextual cues coherent scene transitions emotional tone consistency bridging query residual query end-to-end training AudioStory-10K animated soundscapes natural sound narratives single-audio generation narrative audio generation