GET-Zero: Graph Embodiment Transformer for Zero-shot Embodiment
  Generalization
This paper introduces GET-Zero, a model architecture and training procedure
for learning an embodiment-aware control policy that can immediately adapt to
new hardware changes without retraining. To do so, we present Graph Embodiment
Transformer (GET), a transformer model that leverages the embodiment graph
connectivity as a learned structural bias in the attention mechanism. We use
behavior cloning to distill demonstration data from embodiment-specific expert
policies into an embodiment-aware GET model that conditions on the hardware
configuration of the robot to make control decisions. We conduct a case study
on a dexterous in-hand object rotation task using different configurations of a
four-fingered robot hand with joints removed and with link length extensions.
Using the GET model along with a self-modeling loss enables GET-Zero to
zero-shot generalize to unseen variation in graph structure and link length,
yielding a 20% improvement over baseline methods. All code and qualitative
video results are on https://get-zero-paper.github.io
669f206b352220a858f53375
Austin Patel
admin_assigned
2024-07-23T12:47:31.986Z
65dea524e87db33835824fc2
https://cdn-avatars.huggingface.co/v1/production/uploads/65dea524e87db33835824fc2/8VY4CFwof0kCTpKlPAW3E.jpeg
Austin Patel
user
austinpatel
669f206b352220a858f53376
Shuran Song
GET-Zero: Graph Embodiment Transformer for Zero-shot Embodiment
  Generalization
This paper introduces GET-Zero, a model architecture and training procedure
for learning an embodiment-aware control policy that can immediately adapt to
new hardware changes without retraining. To do so, we present Graph Embodiment
Transformer (GET), a transformer model that leverages the embodiment graph
connectivity as a learned structural bias in the attention mechanism. We use
behavior cloning to distill demonstration data from embodiment-specific expert
policies into an embodiment-aware GET model that conditions on the hardware
configuration of the robot to make control decisions. We conduct a case study
on a dexterous in-hand object rotation task using different configurations of a
four-fingered robot hand with joints removed and with link length extensions.
Using the GET model along with a self-modeling loss enables GET-Zero to
zero-shot generalize to unseen variation in graph structure and link length,
yielding a 20% improvement over baseline methods. All code and qualitative
video results are on https://get-zero-paper.github.io
A Graph Embodiment Transformer model enables a robot to adapt immediately to hardware changes without retraining, demonstrating improvements in zero-shot generalization to unseen configurations.
Graph Embodiment Transformer embodiment graph attention mechanism behavior cloning embodiment-aware hardware configuration self-modeling loss zero-shot generalization dexterous in-hand object rotation four-fingered robot hand