OmniScenes: A Large-scale Simulatable Indoor Scene Dataset with Realistic Layouts
Weipeng Zhong, Peizhou Cao, Yichen Jin, Luo Li, Wenzhe Cai, Jingli Lin, Hanqing Wang, Zhaoyang Lyu, Tai WANG, Xudong XU, Bo Dai, Jiangmiao Pang
The advancement of Embodied AI heavily relies on large-scale, simulatable 3D scene datasets characterized by scene diversity and realistic layouts. However, existing datasets typically suffer from limitations in diversity or simulatability, sanitized layouts lacking small items, and severe object collisions. To address these shortcomings, we introduce \textbf{OmniScenes}, a novel large-scale simulatable indoor scene dataset comprising approximately 40,000 diverse scenes by integrating three disparate scene sources, \ie, real-world scans, procedurally generated scenes, and designer-created scenes, including 1.96M objects and 800k CAD models that cover 15 common scene types and 288 object classes, resulting in complex layouts that have most-ever 41.5 objects per region in average. Our comprehensive data processing pipeline ensures simulatability by creating real-to-sim replicas for real-world scans, achieves realistic layouts by preserving small items, and enhances interactivity by incorporating interactive objects and resolving collisions. We demonstrate the value of OmniScenes with two benchmark applications: scene layout generation and point-goal navigation. Both show the new challenges posed by the complex and realistic layouts. More importantly, OmniScenes paves the way for scaling up the model training for both tasks, making the generation and navigation in such complex scenes possible. We commit to open-sourcing the data, models, and benchmarks to benefit the community.