DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation
Kefei Zhu, Fengshuo Bai, YuanHao Xiang, Yishuai Cai, Xinglin Chen, Ruochong Li, Xingtao Wang, Yaodong Yang, Hao Dong, Xiaopeng Fan, Yuanpei Chen
Dexterous manipulation is critical to advancing robot capabilities in real-world applications, yet diverse and high-quality datasets remain scarce. Existing data collection methods either rely on human teleoperation or require significant human engineering, or are merely limited to grasping, restricting their scalability and generalization. In this paper, we introduce DexFlyWheel, a scalable data generation framework that employs a self-improving cycle to iteratively expand data diversity. Starting from efficient seed demonstrations warmup, our framework expands data diversity via multiple iterations in the self-improving cycle. Each iteration follows a closed-loop pipeline that combines imitation learning, reinforcement learning, rollout trajectory collection, and data augmentation. At each iteration, we first use imitation learning to extract behavioral priors from demonstrations and employ reinforcement learning to enhance generalization. Based on our policy, we rollout trajectories in simulation and then augment these across different environments and objects positions. As iterations progress, our framework generates more diverse data, including various objects, environments, and object positions. Experimental results show that policies trained on our dataset achieve an average success rate of 81.9\% on the challenge test sets, with a real-world transfer success rate of 78.3\% on dual-arm lift tasks. Videos can be found on our project website https://DexFlyWheel.github.io.