Adversarial Data Collection: Human-Collaborative Perturbations for
  Efficient and Robust Robotic Imitation Learning
The pursuit of data efficiency, where quality outweighs quantity, has emerged
as a cornerstone in robotic manipulation, especially given the high costs
associated with real-world data collection. We propose that maximizing the
informational density of individual demonstrations can dramatically reduce
reliance on large-scale datasets while improving task performance. To this end,
we introduce Adversarial Data Collection, a Human-in-the-Loop (HiL) framework
that redefines robotic data acquisition through real-time, bidirectional
human-environment interactions. Unlike conventional pipelines that passively
record static demonstrations, ADC adopts a collaborative perturbation paradigm:
during a single episode, an adversarial operator dynamically alters object
states, environmental conditions, and linguistic commands, while the
tele-operator adaptively adjusts actions to overcome these evolving challenges.
This process compresses diverse failure-recovery behaviors, compositional task
variations, and environmental perturbations into minimal demonstrations. Our
experiments demonstrate that ADC-trained models achieve superior compositional
generalization to unseen task instructions, enhanced robustness to perceptual
perturbations, and emergent error recovery capabilities. Strikingly, models
trained with merely 20% of the demonstration volume collected through ADC
significantly outperform traditional approaches using full datasets. These
advances bridge the gap between data-centric learning paradigms and practical
robotic deployment, demonstrating that strategic data acquisition, not merely
post-hoc processing, is critical for scalable, real-world robot learning.
Additionally, we are curating a large-scale ADC-Robotics dataset comprising
real-world manipulation tasks with adversarial perturbations. This benchmark
will be open-sourced to facilitate advancements in robotic imitation learning.
67d78c194fd0e3fa3a082f8d
Siyuan Huang
claimed_verified
2025-03-17T08:44:21.620Z
634e4120038b5879133552f5
/avatars/34ec861b4bbf1aecf927a7d6e726c7a4.svg
Siyuan
user
SiyuanH
67d78c194fd0e3fa3a082f8e
Yue Liao
admin_assigned
2025-03-17T08:49:27.927Z
670f827bb94a3734d270f707
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/D6qCPBMJAUgozfG7YTwky.png
Yue Liao
user
morninghaze
67d78c194fd0e3fa3a082f8f
Siyuan Feng
claimed_verified
2025-03-17T08:44:19.837Z
620326e962b2b0e46e79971b
https://cdn-avatars.huggingface.co/v1/production/uploads/620326e962b2b0e46e79971b/1FVPRpsWng5q3An4qbuYQ.jpeg
Siyuan Feng
user
Eralien
67d78c194fd0e3fa3a082f90
Shu Jiang
claimed_verified
2025-03-17T14:42:21.098Z
666c463389e21df7d4a34d03
/avatars/8f73e78d740ace263961de1a2896fc09.svg
姜姝
user
jiang12345
67d78c194fd0e3fa3a082f91
Si Liu
67d78c194fd0e3fa3a082f92
Hongsheng Li
admin_assigned
2025-03-17T08:49:51.674Z
65c04e9c27a5fdca81abcbd9
/avatars/12a155683c824fa23da4a9e2bed4f64e.svg
Hongsheng LI
user
hsli-cuhk
67d78c194fd0e3fa3a082f93
Maoqing Yao
admin_assigned
2025-03-17T08:49:59.798Z
67739bfa64e8b7438ae68eb4
/avatars/15193bfbce487b2de4ce8c86bd18885a.svg
Maoqing Yao
user
AutobotZero
67d78c194fd0e3fa3a082f94
Guanghui Ren
admin_assigned
2025-03-17T08:50:05.432Z
646ec9b135f55eb49e405faa
/avatars/a17194be585d20e2a021e77a5a20e213.svg
Guanghui Ren
user
sundrops
Adversarial Data Collection: Human-Collaborative Perturbations for
  Efficient and Robust Robotic Imitation Learning
The pursuit of data efficiency, where quality outweighs quantity, has emerged
as a cornerstone in robotic manipulation, especially given the high costs
associated with real-world data collection. We propose that maximizing the
informational density of individual demonstrations can dramatically reduce
reliance on large-scale datasets while improving task performance. To this end,
we introduce Adversarial Data Collection, a Human-in-the-Loop (HiL) framework
that redefines robotic data acquisition through real-time, bidirectional
human-environment interactions. Unlike conventional pipelines that passively
record static demonstrations, ADC adopts a collaborative perturbation paradigm:
during a single episode, an adversarial operator dynamically alters object
states, environmental conditions, and linguistic commands, while the
tele-operator adaptively adjusts actions to overcome these evolving challenges.
This process compresses diverse failure-recovery behaviors, compositional task
variations, and environmental perturbations into minimal demonstrations. Our
experiments demonstrate that ADC-trained models achieve superior compositional
generalization to unseen task instructions, enhanced robustness to perceptual
perturbations, and emergent error recovery capabilities. Strikingly, models
trained with merely 20% of the demonstration volume collected through ADC
significantly outperform traditional approaches using full datasets. These
advances bridge the gap between data-centric learning paradigms and practical
robotic deployment, demonstrating that strategic data acquisition, not merely
post-hoc processing, is critical for scalable, real-world robot learning.
Additionally, we are curating a large-scale ADC-Robotics dataset comprising
real-world manipulation tasks with adversarial perturbations. This benchmark
will be open-sourced to facilitate advancements in robotic imitation learning.
Adversarial Data Collection, a Human-in-the-Loop framework that uses dynamic perturbations, reduces data needs while enhancing the performance and robustness of robotic manipulation models.
Adversarial Data Collection Human-in-the-Loop HiL bidirectional interactions adversarial operator tele-operator failure-recovery behaviors compositional task variations perceptual perturbations error recovery capabilities robotic manipulation imitation learning