Sekai: A Video Dataset for World Exploration
Zhen Li, Chuanhao Li, Xiaofeng Mao, Shaoheng Lin, Ming Li, Shitian Zhao, Zhaopan Xu, Xinyue Li, Yukang Feng, Jianwen Sun, Zizhen Li, Fanrui Zhang, Jiaxin Ai, Zhixiang Wang, Yuwei Wu, Tong He, Yunde Jia, Kaipeng Zhang
World exploration is an important human activity to forms the foundation of humankind's odyssey, while video generation techniques have made remarkable progress, promising to be the foundation of interactive world exploration.However, existing video generation datasets are not well-suited for world exploration training as they suffer from some limitations: limited duration, static scenes, and lack of exploratory and world annotations.In this paper, we introduce Sekai せかい, meaning "world'' in Japanese), a high-quality first-person view worldwide video dataset with rich annotations for world exploration. It consists of over 6,000 hours of walking or drone view (FPV and UVA) videos from 65 countries, over 1000 cities. We develop an efficient toolchain to pre-process videos and an effective toolbox to annotate them. For each video, we annotate location, scene type, weather, crowd density, captions, and camera trajectories.Experiments demonstrate the quality and effectiveness of the dataset. We believe Sekai will benefit the area of video generation and world exploration, and raise valuable applications. We make an introductory video about the dataset in the supplemental material.