ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of
  Video
The Internet's wealth of content, with up to 60% published in English,
starkly contrasts the global population, where only 18.8% are English speakers,
and just 5.1% consider it their native language, leading to disparities in
online information access. Unfortunately, automated processes for dubbing of
video - replacing the audio track of a video with a translated alternative -
remains a complex and challenging task due to pipelines, necessitating precise
timing, facial movement synchronization, and prosody matching. While end-to-end
dubbing offers a solution, data scarcity continues to impede the progress of
both end-to-end and pipeline-based methods. In this work, we introduce
Anim-400K, a comprehensive dataset of over 425K aligned animated video segments
in Japanese and English supporting various video-related tasks, including
automated dubbing, simultaneous translation, guided video summarization, and
genre/theme/style classification. Our dataset is made publicly available for
research purposes at https://github.com/davidmchan/Anim400K.
659f6fbec0b775ab966257dd
Kevin Cai
659f6fbec0b775ab966257de
Chonghua Liu
admin_assigned
2024-01-11T11:18:43.130Z
650efa2457343388e5773b67
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/E8ndPzmtufhO9tvr4GsPi.jpeg
Chonghua Liu
user
mliu0515
659f6fbec0b775ab966257df
David M. Chan
extracted_confirmed
2024-01-11T06:35:30.606Z
6388f68c43d8b0797a09ff84
https://cdn-avatars.huggingface.co/v1/production/uploads/1669920369989-noauth.jpeg
David Chan
user
davidchan
ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of
  Video
The Internet's wealth of content, with up to 60% published in English,
starkly contrasts the global population, where only 18.8% are English speakers,
and just 5.1% consider it their native language, leading to disparities in
online information access. Unfortunately, automated processes for dubbing of
video - replacing the audio track of a video with a translated alternative -
remains a complex and challenging task due to pipelines, necessitating precise
timing, facial movement synchronization, and prosody matching. While end-to-end
dubbing offers a solution, data scarcity continues to impede the progress of
both end-to-end and pipeline-based methods. In this work, we introduce
Anim-400K, a comprehensive dataset of over 425K aligned animated video segments
in Japanese and English supporting various video-related tasks, including
automated dubbing, simultaneous translation, guided video summarization, and
genre/theme/style classification. Our dataset is made publicly available for
research purposes at https://github.com/davidmchan/Anim400K.
Anim-400K is a large-scale dataset for various video-related tasks, including automated dubbing, with over 425K aligned segments in Japanese and English.
end-to-end dubbing pipeline-based methods simultaneous translation guided video summarization genre/theme/style classification