Bootstrapping Language-Guided Navigation Learning with Self-Refining
  Data Flywheel
Creating high-quality data for training robust language-instructed agents is
a long-lasting challenge in embodied AI. In this paper, we introduce a
Self-Refining Data Flywheel (SRDF) that generates high-quality and large-scale
navigational instruction-trajectory pairs by iteratively refining the data pool
through the collaboration between two models, the instruction generator and the
navigator, without any human-in-the-loop annotation. Specifically, SRDF starts
with using a base generator to create an initial data pool for training a base
navigator, followed by applying the trained navigator to filter the data pool.
This leads to higher-fidelity data to train a better generator, which can, in
turn, produce higher-quality data for training the next-round navigator. Such a
flywheel establishes a data self-refining process, yielding a continuously
improved and highly effective dataset for large-scale language-guided
navigation learning. Our experiments demonstrate that after several flywheel
rounds, the navigator elevates the performance boundary from 70% to 78% SPL on
the classic R2R test set, surpassing human performance (76%) for the first
time. Meanwhile, this process results in a superior generator, evidenced by a
SPICE increase from 23.5 to 26.2, better than all previous VLN instruction
generation methods. Finally, we demonstrate the scalability of our method
through increasing environment and instruction diversity, and the
generalization ability of our pre-trained navigator across various downstream
navigation tasks, surpassing state-of-the-art methods by a large margin in all
cases.
675ad589659e5553260b6610
Zun Wang
admin_assigned
2024-12-16T13:54:52.948Z
6622c710b0e5c5e3de8311c1
/avatars/a824c150040731679bbd77762ca9d4eb.svg
Zun Wang
user
ZunWang
675ad589659e5553260b6611
Jialu Li
admin_assigned
2024-12-16T13:54:45.022Z
65d69eb0bcf1fed72325a052
/avatars/5ef735c3137555bb52e512ae820e437c.svg
Jialu Li
user
jialuliluka
675ad589659e5553260b6612
Yicong Hong
675ad589659e5553260b6613
Songze Li
claimed_verified
2025-07-03T16:19:54.516Z
64acbbd51aee69ece03c6c0c
/avatars/604df1cabc5faeda55022ae4c1997e56.svg
Songze Li
user
SongzeLi
675ad589659e5553260b6614
Kunchang Li
admin_assigned
2024-12-16T13:53:39.201Z
61fb81006374891646732f37
https://cdn-avatars.huggingface.co/v1/production/uploads/1643872995181-61fb81006374891646732f37.jpeg
Kunchang Li
user
Andy1621
675ad589659e5553260b6615
Shoubin Yu
admin_assigned
2024-12-16T13:54:02.398Z
63ee8fb05f1300034de097fd
/avatars/bceedb88927d8633948266503c2dd0b1.svg
Yu
user
Shoubin
675ad589659e5553260b6616
Yi Wang
675ad589659e5553260b6617
Yu Qiao
675ad589659e5553260b6618
Yali Wang
675ad589659e5553260b6619
Mohit Bansal
admin_assigned
2024-12-16T13:54:19.115Z
665d9d3a057f7c508f98c625
https://cdn-avatars.huggingface.co/v1/production/uploads/665d9d3a057f7c508f98c625/u1R9P9sJoAl4zEIcetbPy.jpeg
Mohit Bansal
user
mohitbansal
675ad589659e5553260b661a
Limin Wang
Bootstrapping Language-Guided Navigation Learning with Self-Refining
  Data Flywheel
Creating high-quality data for training robust language-instructed agents is
a long-lasting challenge in embodied AI. In this paper, we introduce a
Self-Refining Data Flywheel (SRDF) that generates high-quality and large-scale
navigational instruction-trajectory pairs by iteratively refining the data pool
through the collaboration between two models, the instruction generator and the
navigator, without any human-in-the-loop annotation. Specifically, SRDF starts
with using a base generator to create an initial data pool for training a base
navigator, followed by applying the trained navigator to filter the data pool.
This leads to higher-fidelity data to train a better generator, which can, in
turn, produce higher-quality data for training the next-round navigator. Such a
flywheel establishes a data self-refining process, yielding a continuously
improved and highly effective dataset for large-scale language-guided
navigation learning. Our experiments demonstrate that after several flywheel
rounds, the navigator elevates the performance boundary from 70% to 78% SPL on
the classic R2R test set, surpassing human performance (76%) for the first
time. Meanwhile, this process results in a superior generator, evidenced by a
SPICE increase from 23.5 to 26.2, better than all previous VLN instruction
generation methods. Finally, we demonstrate the scalability of our method
through increasing environment and instruction diversity, and the
generalization ability of our pre-trained navigator across various downstream
navigation tasks, surpassing state-of-the-art methods by a large margin in all
cases.
A self-refining data flywheel generates high-quality navigational instruction-trajectory pairs for training language-instructed agents, surpassing human performance in navigation tasks.
Self-Refining Data Flywheel SRDF instruction generator navigator data self-refining process SPL R2R test set SPICE VLN instruction generation scalability generalization ability