UniT: Unified Tactile Representation for Robot Learning
UniT is a novel approach to tactile representation learning, using VQVAE to
learn a compact latent space and serve as the tactile representation. It uses
tactile images obtained from a single simple object to train the representation
with transferability and generalizability. This tactile representation can be
zero-shot transferred to various downstream tasks, including perception tasks
and manipulation policy learning. Our benchmarking on an in-hand 3D pose
estimation task shows that UniT outperforms existing visual and tactile
representation learning methods. Additionally, UniT's effectiveness in policy
learning is demonstrated across three real-world tasks involving diverse
manipulated objects and complex robot-object-environment interactions. Through
extensive experimentation, UniT is shown to be a simple-to-train,
plug-and-play, yet widely effective method for tactile representation learning.
For more details, please refer to our open-source repository
https://github.com/ZhengtongXu/UniT and the project website
https://zhengtongxu.github.io/unifiedtactile.github.io/.
66bc26c5f36eed1977865083
Zhengtong Xu
claimed_verified
2024-08-15T10:11:53.296Z
66bcfb28dac7d713269cbfd2
/avatars/1b20fefdd0cf5a7ceda515b361d56016.svg
Zhengtong Xu
user
Zhengtongxu
66bc26c5f36eed1977865084
Raghava Uppuluri
admin_assigned
2024-08-14T11:06:37.737Z
60b25904295203cd10efe5c1
/avatars/d8062f3dabed336b530c7c05a61e2868.svg
Raghava Uppuluri
user
raghavauppuluri
66bc26c5f36eed1977865085
Xinwei Zhang
claimed_verified
2024-08-15T10:11:41.534Z
646fdb33cc049b49e35ff10c
/avatars/53606e9b66e721a70be4b392a2ce19b3.svg
Xinwei Zhang
user
Cyruzzz
66bc26c5f36eed1977865086
Cael Fitch
66bc26c5f36eed1977865087
Philip Glen Crandall
66bc26c5f36eed1977865088
Wan Shou
66bc26c5f36eed1977865089
Dongyi Wang
admin_assigned
2024-08-14T11:07:26.310Z
6660dc7bc629e0e089e67468
/avatars/6878288c695bece4bb75c02bb85c7e49.svg
Dongyi Wang
user
DongyiWang
66bc26c5f36eed197786508a
Yu She
UniT: Unified Tactile Representation for Robot Learning
UniT is a novel approach to tactile representation learning, using VQVAE to
learn a compact latent space and serve as the tactile representation. It uses
tactile images obtained from a single simple object to train the representation
with transferability and generalizability. This tactile representation can be
zero-shot transferred to various downstream tasks, including perception tasks
and manipulation policy learning. Our benchmarking on an in-hand 3D pose
estimation task shows that UniT outperforms existing visual and tactile
representation learning methods. Additionally, UniT's effectiveness in policy
learning is demonstrated across three real-world tasks involving diverse
manipulated objects and complex robot-object-environment interactions. Through
extensive experimentation, UniT is shown to be a simple-to-train,
plug-and-play, yet widely effective method for tactile representation learning.
For more details, please refer to our open-source repository
https://github.com/ZhengtongXu/UniT and the project website
https://zhengtongxu.github.io/unifiedtactile.github.io/.
UniT uses VQVAE for tactile representation learning, demonstrating superior performance in various tasks including pose estimation and policy learning, with zero-shot transfer capabilities.
VQVAE tactile representation latent space zero-shot transfer in-hand 3D pose estimation perception tasks manipulation policy learning