Lessons from Learning to Spin "Pens"
In-hand manipulation of pen-like objects is an important skill in our daily
lives, as many tools such as hammers and screwdrivers are similarly shaped.
However, current learning-based methods struggle with this task due to a lack
of high-quality demonstrations and the significant gap between simulation and
the real world. In this work, we push the boundaries of learning-based in-hand
manipulation systems by demonstrating the capability to spin pen-like objects.
We first use reinforcement learning to train an oracle policy with privileged
information and generate a high-fidelity trajectory dataset in simulation. This
serves two purposes: 1) pre-training a sensorimotor policy in simulation; 2)
conducting open-loop trajectory replay in the real world. We then fine-tune the
sensorimotor policy using these real-world trajectories to adapt it to the real
world dynamics. With less than 50 trajectories, our policy learns to rotate
more than ten pen-like objects with different physical properties for multiple
revolutions. We present a comprehensive analysis of our design choices and
share the lessons learned during development.
66a701400651bc96877907f9
Jun Wang
claimed_verified
2024-07-29T14:58:05.196Z
6406fdc42e309e65451deb6b
/avatars/c94326a5746330a3ad050b9c4191f4eb.svg
Jun Wang
user
Junw048
66a701400651bc96877907fa
Ying Yuan
claimed_verified
2024-07-29T14:58:06.827Z
66a78bbad794ddcd151441fb
/avatars/210f139d0c40d4c051deb46a4477d073.svg
Ying Yuan
user
YingYuan0414
66a701400651bc96877907fb
Haichuan Che
66a701400651bc96877907fc
Haozhi Qi
admin_assigned
2024-07-29T08:52:07.920Z
658871e7d1331d552be8b457
/avatars/c323178ad1e0fbdfa5e66dcd4ba357e6.svg
Haozhi Qi
user
HaozhiQi
66a701400651bc96877907fd
Yi Ma
66a701400651bc96877907fe
Jitendra Malik
admin_assigned
2024-07-29T08:52:24.489Z
65369a95605a07338de78ab0
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sGFjOjLT2akN-sn5beVWL.jpeg
Jitendra Malik 
user
jitendra1995
66a701400651bc96877907ff
Xiaolong Wang
Lessons from Learning to Spin "Pens"
In-hand manipulation of pen-like objects is an important skill in our daily
lives, as many tools such as hammers and screwdrivers are similarly shaped.
However, current learning-based methods struggle with this task due to a lack
of high-quality demonstrations and the significant gap between simulation and
the real world. In this work, we push the boundaries of learning-based in-hand
manipulation systems by demonstrating the capability to spin pen-like objects.
We first use reinforcement learning to train an oracle policy with privileged
information and generate a high-fidelity trajectory dataset in simulation. This
serves two purposes: 1) pre-training a sensorimotor policy in simulation; 2)
conducting open-loop trajectory replay in the real world. We then fine-tune the
sensorimotor policy using these real-world trajectories to adapt it to the real
world dynamics. With less than 50 trajectories, our policy learns to rotate
more than ten pen-like objects with different physical properties for multiple
revolutions. We present a comprehensive analysis of our design choices and
share the lessons learned during development.
A reinforcement learning-based approach trains a sensorimotor policy for in-hand manipulation of pen-like objects through simulation and real-world fine-tuning with limited trajectories.
reinforcement learning oracle policy high-fidelity trajectory dataset sensorimotor policy open-loop trajectory replay real-world dynamics fine-tuning in-hand manipulation