Do You Need Proprioceptive States in Visuomotor Policies?
Imitation-learning-based visuomotor policies have been widely used in robot
manipulation, where both visual observations and proprioceptive states are
typically adopted together for precise control. However, in this study, we find
that this common practice makes the policy overly reliant on the proprioceptive
state input, which causes overfitting to the training trajectories and results
in poor spatial generalization. On the contrary, we propose the State-free
Policy, removing the proprioceptive state input and predicting actions only
conditioned on visual observations. The State-free Policy is built in the
relative end-effector action space, and should ensure the full task-relevant
visual observations, here provided by dual wide-angle wrist cameras. Empirical
results demonstrate that the State-free policy achieves significantly stronger
spatial generalization than the state-based policy: in real-world tasks such as
pick-and-place, challenging shirt-folding, and complex whole-body manipulation,
spanning multiple robot embodiments, the average success rate improves from 0\%
to 85\% in height generalization and from 6\% to 64\% in horizontal
generalization. Furthermore, they also show advantages in data efficiency and
cross-embodiment adaptation, enhancing their practicality for real-world
deployment.
68d387b50e215259d193b364
Juntu Zhao
claimed_verified
2025-09-24T13:52:57.974Z
63217b064b3a874743797fa5
https://cdn-avatars.huggingface.co/v1/production/uploads/63217b064b3a874743797fa5/FBaU2EiZFO2R_6alDcox8.png
JT Zhao
user
JTZhaoSJTU
68d387b50e215259d193b365
Wenbo Lu
claimed_verified
2025-09-25T07:11:30.104Z
660cb8813134c1a046eaf815
/avatars/45a2e18eb95a06ce3b0740ae2589cb4e.svg
Wenbo Lu
user
fruitswordman
68d387b50e215259d193b366
Di Zhang
68d387b50e215259d193b367
Yufeng Liu
claimed_verified
2025-09-24T13:52:54.957Z
65d72c946a36b5b354f80cf8
/avatars/2cfc99ccd4f52f4a60c7fa40fe4313b7.svg
lyfeng
user
lyfeng001
68d387b50e215259d193b368
Yushen Liang
admin_assigned
2025-09-24T15:26:00.067Z
683fb536ef97de05eb2b62f2
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/cAO8ffm_uin1me1zVdTuk.png
Yushen Liang
user
Mirage415
68d387b50e215259d193b369
Tianluo Zhang
68d387b50e215259d193b36a
Yifeng Cao
admin_assigned
2025-09-24T15:26:18.034Z
68d38f3398cf2f221b971293
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/xk5Ew0vVsPyEY0fqirsSO.png
Yifeng Cao
user
MattC401
68d387b50e215259d193b36b
Junyuan Xie
admin_assigned
2025-09-24T15:26:25.322Z
67cfea72ad91643b5cb92b26
/avatars/11937ee700bf6a743cb627928bebd214.svg
Junyuan Xie
user
piiswrong
68d387b50e215259d193b36c
Yingdong Hu
68d387b50e215259d193b36d
Shengjie Wang
68d387b50e215259d193b36e
Junliang Guo
admin_assigned
2025-09-24T15:26:42.060Z
66868659ccb9539da85c4e14
/avatars/515a49363872c23d57a6f75063606348.svg
Junliang Guo
user
leo-guo
68d387b50e215259d193b36f
Dequan Wang
claimed_verified
2025-09-24T13:52:50.291Z
61ad24836da53246bd6ac410
https://cdn-avatars.huggingface.co/v1/production/uploads/61ad24836da53246bd6ac410/o-FL-C6B77iB94wyAtTuO.png
Dequan Wang
user
dqwang
68d387b50e215259d193b370
Yang Gao
Do You Need Proprioceptive States in Visuomotor Policies?
Imitation-learning-based visuomotor policies have been widely used in robot
manipulation, where both visual observations and proprioceptive states are
typically adopted together for precise control. However, in this study, we find
that this common practice makes the policy overly reliant on the proprioceptive
state input, which causes overfitting to the training trajectories and results
in poor spatial generalization. On the contrary, we propose the State-free
Policy, removing the proprioceptive state input and predicting actions only
conditioned on visual observations. The State-free Policy is built in the
relative end-effector action space, and should ensure the full task-relevant
visual observations, here provided by dual wide-angle wrist cameras. Empirical
results demonstrate that the State-free policy achieves significantly stronger
spatial generalization than the state-based policy: in real-world tasks such as
pick-and-place, challenging shirt-folding, and complex whole-body manipulation,
spanning multiple robot embodiments, the average success rate improves from 0\%
to 85\% in height generalization and from 6\% to 64\% in horizontal
generalization. Furthermore, they also show advantages in data efficiency and
cross-embodiment adaptation, enhancing their practicality for real-world
deployment.
A state-free policy using only visual observations achieves better spatial generalization and data efficiency in robot manipulation tasks compared to state-based policies.
imitation-learning-based visuomotor policies proprioceptive state input overfitting spatial generalization relative end-effector action space dual wide-angle wrist cameras pick-and-place shirt-folding whole-body manipulation cross-embodiment adaptation