PoseLess: Depth-Free Vision-to-Joint Control via Direct Image Mapping
  with VLM
This paper introduces PoseLess, a novel framework for robot hand control that
eliminates the need for explicit pose estimation by directly mapping 2D images
to joint angles using projected representations. Our approach leverages
synthetic training data generated through randomized joint configurations,
enabling zero-shot generalization to real-world scenarios and cross-morphology
transfer from robotic to human hands. By projecting visual inputs and employing
a transformer-based decoder, PoseLess achieves robust, low-latency control
while addressing challenges such as depth ambiguity and data scarcity.
Experimental results demonstrate competitive performance in joint angle
prediction accuracy without relying on any human-labelled dataset.
67d4d78897767f49253a0739
Alan Dao
claimed_verified
2025-03-24T16:39:52.798Z
62d7b2339b629105a5d6888a
https://cdn-avatars.huggingface.co/v1/production/uploads/62d7b2339b629105a5d6888a/pYi994fqzXctgoed79oUo.png
Alan Dao
user
alandao
67d4d78897767f49253a073a
Dinh Bach Vu
67d4d78897767f49253a073b
Tuan Le Duc Anh
67d4d78897767f49253a073c
Bui Quang Huy
PoseLess: Depth-Free Vision-to-Joint Control via Direct Image Mapping
  with VLM
This paper introduces PoseLess, a novel framework for robot hand control that
eliminates the need for explicit pose estimation by directly mapping 2D images
to joint angles using projected representations. Our approach leverages
synthetic training data generated through randomized joint configurations,
enabling zero-shot generalization to real-world scenarios and cross-morphology
transfer from robotic to human hands. By projecting visual inputs and employing
a transformer-based decoder, PoseLess achieves robust, low-latency control
while addressing challenges such as depth ambiguity and data scarcity.
Experimental results demonstrate competitive performance in joint angle
prediction accuracy without relying on any human-labelled dataset.
PoseLess uses synthetic training data and transformer-based decoders to map 2D images to joint angles for robot hand control, achieving zero-shot generalization to real-world scenarios and cross-morphology transfer.
transformer-based decoder