In-Context Prompt Editing For Conditional Audio Generation
Distributional shift is a central challenge in the deployment of machine
learning models as they can be ill-equipped for real-world data. This is
particularly evident in text-to-audio generation where the encoded
representations are easily undermined by unseen prompts, which leads to the
degradation of generated audio -- the limited set of the text-audio pairs
remains inadequate for conditional audio generation in the wild as user prompts
are under-specified. In particular, we observe a consistent audio quality
degradation in generated audio samples with user prompts, as opposed to
training set prompts. To this end, we present a retrieval-based in-context
prompt editing framework that leverages the training captions as demonstrative
exemplars to revisit the user prompts. We show that the framework enhanced the
audio quality across the set of collected user prompts, which were edited with
reference to the training captions as exemplars.
6545070ca28c87edcfa1ac01
Ernie Chang
6545070ca28c87edcfa1ac02
Pin-Jie Lin
admin_assigned
2023-11-03T15:01:17.151Z
61c6133ed9ca62b30bb7726e
/avatars/74471b38c62fef4233b34045868c71cf.svg
Pin-Jie Lin
user
linusdd44804
6545070ca28c87edcfa1ac03
Yang Li
claimed_verified
2023-11-07T11:20:49.631Z
654971d49b86bd6b2c6216ee
/avatars/90fc2319c6d67694d4cd13401d9d2db3.svg
Yang Li
user
jerryyangli
6545070ca28c87edcfa1ac04
Sidd Srinivasan
admin_assigned
2023-11-03T15:06:33.331Z
64e23052c3310dd6cc5e0f38
/avatars/860a3cb6aedf722f1ff0e93b4c213e18.svg
Sidd Srinivasan
user
Sidds
6545070ca28c87edcfa1ac05
Gael Le Lan
6545070ca28c87edcfa1ac06
David Kant
6545070ca28c87edcfa1ac07
Yangyang Shi
6545070ca28c87edcfa1ac08
Forrest Iandola
admin_assigned
2023-11-03T15:07:16.917Z
5f1753f4925b9863e28ad4de
/avatars/e56d25a40f2c0e4e6696b9738877fce8.svg
Forrest Iandola
user
forresti
6545070ca28c87edcfa1ac09
Vikas Chandra
In-Context Prompt Editing For Conditional Audio Generation
Distributional shift is a central challenge in the deployment of machine
learning models as they can be ill-equipped for real-world data. This is
particularly evident in text-to-audio generation where the encoded
representations are easily undermined by unseen prompts, which leads to the
degradation of generated audio -- the limited set of the text-audio pairs
remains inadequate for conditional audio generation in the wild as user prompts
are under-specified. In particular, we observe a consistent audio quality
degradation in generated audio samples with user prompts, as opposed to
training set prompts. To this end, we present a retrieval-based in-context
prompt editing framework that leverages the training captions as demonstrative
exemplars to revisit the user prompts. We show that the framework enhanced the
audio quality across the set of collected user prompts, which were edited with
reference to the training captions as exemplars.
A retrieval-based framework enhances audio quality in text-to-audio generation by revisiting and editing user prompts with reference to training exemplars.
text-to-audio generation encoded representations conditional audio generation distributional shift retrieval-based framework in-context prompt editing training captions generated audio quality