EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video
  Generation
Video generation has emerged as a promising tool for world simulation,
leveraging visual data to replicate real-world environments. Within this
context, egocentric video generation, which centers on the human perspective,
holds significant potential for enhancing applications in virtual reality,
augmented reality, and gaming. However, the generation of egocentric videos
presents substantial challenges due to the dynamic nature of egocentric
viewpoints, the intricate diversity of actions, and the complex variety of
scenes encountered. Existing datasets are inadequate for addressing these
challenges effectively. To bridge this gap, we present EgoVid-5M, the first
high-quality dataset specifically curated for egocentric video generation.
EgoVid-5M encompasses 5 million egocentric video clips and is enriched with
detailed action annotations, including fine-grained kinematic control and
high-level textual descriptions. To ensure the integrity and usability of the
dataset, we implement a sophisticated data cleaning pipeline designed to
maintain frame consistency, action coherence, and motion smoothness under
egocentric conditions. Furthermore, we introduce EgoDreamer, which is capable
of generating egocentric videos driven simultaneously by action descriptions
and kinematic control signals. The EgoVid-5M dataset, associated action
annotations, and all data cleansing metadata will be released for the
advancement of research in egocentric video generation.
67357e302a3c30eac4c75625
Xiaofeng Wang
claimed_verified
2024-11-15T09:21:57.290Z
6426616ea5ec4a5cbc535634
https://cdn-avatars.huggingface.co/v1/production/uploads/6426616ea5ec4a5cbc535634/hH6JsxnXeakH3mBTeNNmO.jpeg
JeffWang
user
Jeff-Wang
67357e302a3c30eac4c75626
Kang Zhao
67357e302a3c30eac4c75627
Feng Liu
claimed_verified
2024-12-02T09:33:47.443Z
661658b9e398e4f0757221ce
/avatars/9e9d55145661d00e2a9e029ba2e085d8.svg
Liu Feng
user
LiewFeng
67357e302a3c30eac4c75628
Jiayu Wang
admin_assigned
2024-11-15T14:15:26.293Z
64ec8ce1a91f5a573e4ab454
/avatars/1c855b26abe040a4d19ade2be29fc1c2.svg
Jiayu Wang
user
ArseneRean
67357e302a3c30eac4c75629
Guosheng Zhao
admin_assigned
2024-11-15T14:14:35.900Z
654c7f32386fc5525c1ebc12
/avatars/d2cf2c7c481e6e6f5d15056f1ec0ef10.svg
zhao guosheng
user
f1yfish
67357e302a3c30eac4c7562a
Xiaoyi Bao
admin_assigned
2024-11-15T14:14:19.541Z
6615fc38d6af2bbe8bd86a72
/avatars/812277f2f5fd673f2e18988cd499c74d.svg
Xiaoyi Bao
user
Shawnee-bxy
67357e302a3c30eac4c7562b
Zheng Zhu
admin_assigned
2024-11-15T14:14:10.066Z
656e9b562cd7a3e348011d26
/avatars/bcca51bdc27c664f8f132420e6ed99fa.svg
Zheng Zhu
user
ZhengZhu
67357e302a3c30eac4c7562c
Yingya Zhang
67357e302a3c30eac4c7562d
Xingang Wang
admin_assigned
2024-11-15T14:13:53.462Z
64f8118bc3c12b377c976dd0
/avatars/0bb606bba6d8e91745be66eb9942cad1.svg
Xingang Wang
user
purityWang
EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video
  Generation
Video generation has emerged as a promising tool for world simulation,
leveraging visual data to replicate real-world environments. Within this
context, egocentric video generation, which centers on the human perspective,
holds significant potential for enhancing applications in virtual reality,
augmented reality, and gaming. However, the generation of egocentric videos
presents substantial challenges due to the dynamic nature of egocentric
viewpoints, the intricate diversity of actions, and the complex variety of
scenes encountered. Existing datasets are inadequate for addressing these
challenges effectively. To bridge this gap, we present EgoVid-5M, the first
high-quality dataset specifically curated for egocentric video generation.
EgoVid-5M encompasses 5 million egocentric video clips and is enriched with
detailed action annotations, including fine-grained kinematic control and
high-level textual descriptions. To ensure the integrity and usability of the
dataset, we implement a sophisticated data cleaning pipeline designed to
maintain frame consistency, action coherence, and motion smoothness under
egocentric conditions. Furthermore, we introduce EgoDreamer, which is capable
of generating egocentric videos driven simultaneously by action descriptions
and kinematic control signals. The EgoVid-5M dataset, associated action
annotations, and all data cleansing metadata will be released for the
advancement of research in egocentric video generation.
EgoDreamer generates egocentric videos using EgoVid-5M, a new high-quality dataset with detailed action annotations and kinematic controls, enhancing virtual reality, augmented reality, and gaming applications.
egocentric video generation EgoVid-5M action descriptions kinematic control signals data cleaning pipeline frame consistency action coherence motion smoothness EgoDreamer