Fidelity-Aware Data Composition for Robust Robot Generalization
Generalist robot policies trained on large-scale, visually homogeneous
datasets can be susceptible to shortcut learning, which impairs their
out-of-distribution (OOD) generalization. While generative data augmentation is
a common approach to introduce diversity, it presents a subtle challenge: data
composition. Naively mixing real and synthetic data can corrupt the learning
signal, as this process often prioritizes visual diversity at the expense of
information fidelity. This paper suggests that robust generalization depends on
principled, fidelity-aware data composition. We introduce Coherent Information
Fidelity Tuning (CIFT), a framework that treats data composition as an
optimization problem. CIFT uses a practical proxy for Information Fidelity
based on the feature-space geometry of a dataset. This enables the
identification of a phase transition, termed the Decoherence Point, where
training stability degrades. The framework includes a generative engine,
Multi-View Video Augmentation (MVAug), to synthesize a causally disentangled
data spectrum for this tuning process. Applying CIFT to policy architectures
such as pi_0 and Diffusion Policy improves OOD success rates by over 54\%.
These results indicate that fidelity-aware composition, beyond data synthesis
alone, is an important component for developing robust, general-purpose robots.
68e8bc3095e8e6771df38bd8
Zizhao Tong
68e8bc3095e8e6771df38bd9
Di Chen
68e8bc3095e8e6771df38bda
Sicheng Hu
68e8bc3095e8e6771df38bdb
Hongwei Fan
68e8bc3095e8e6771df38bdc
Liliang Chen
68e8bc3095e8e6771df38bdd
Guanghui Ren
68e8bc3095e8e6771df38bde
Hao Tang
68e8bc3095e8e6771df38bdf
Hao Dong
68e8bc3095e8e6771df38be0
Ling Shao
Fidelity-Aware Data Composition for Robust Robot Generalization
Generalist robot policies trained on large-scale, visually homogeneous
datasets can be susceptible to shortcut learning, which impairs their
out-of-distribution (OOD) generalization. While generative data augmentation is
a common approach to introduce diversity, it presents a subtle challenge: data
composition. Naively mixing real and synthetic data can corrupt the learning
signal, as this process often prioritizes visual diversity at the expense of
information fidelity. This paper suggests that robust generalization depends on
principled, fidelity-aware data composition. We introduce Coherent Information
Fidelity Tuning (CIFT), a framework that treats data composition as an
optimization problem. CIFT uses a practical proxy for Information Fidelity
based on the feature-space geometry of a dataset. This enables the
identification of a phase transition, termed the Decoherence Point, where
training stability degrades. The framework includes a generative engine,
Multi-View Video Augmentation (MVAug), to synthesize a causally disentangled
data spectrum for this tuning process. Applying CIFT to policy architectures
such as pi_0 and Diffusion Policy improves OOD success rates by over 54\%.
These results indicate that fidelity-aware composition, beyond data synthesis
alone, is an important component for developing robust, general-purpose robots.
Coherent Information Fidelity Tuning (CIFT) improves out-of-distribution generalization in robot policies by optimizing data composition with a generative engine, enhancing robustness and performance.
shortcut learning out-of-distribution (OOD) generalization generative data augmentation data composition Information Fidelity feature-space geometry Decoherence Point Multi-View Video Augmentation (MVAug) causally disentangled policy architectures Diffusion Policy