From Text to Motion: Grounding GPT-4 in a Humanoid Robot "Alter3"
We report the development of Alter3, a humanoid robot capable of generating
spontaneous motion using a Large Language Model (LLM), specifically GPT-4. This
achievement was realized by integrating GPT-4 into our proprietary android,
Alter3, thereby effectively grounding the LLM with Alter's bodily movement.
Typically, low-level robot control is hardware-dependent and falls outside the
scope of LLM corpora, presenting challenges for direct LLM-based robot control.
However, in the case of humanoid robots like Alter3, direct control is feasible
by mapping the linguistic expressions of human actions onto the robot's body
through program code. Remarkably, this approach enables Alter3 to adopt various
poses, such as a 'selfie' stance or 'pretending to be a ghost,' and generate
sequences of actions over time without explicit programming for each body part.
This demonstrates the robot's zero-shot learning capabilities. Additionally,
verbal feedback can adjust poses, obviating the need for fine-tuning. A video
of Alter3's generated motions is available at
https://tnoinkwms.github.io/ALTER-LLM/
6577ce7f9c5c26f96ce634ee
Takahide Yoshida
claimed_verified
2023-12-12T16:23:29.583Z
6578307176b6de7978ffeee7
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/wRf2mV0sIGIDzdrwkH6nC.png
Takahide Yoshida
user
tnoinkwms
6577ce7f9c5c26f96ce634ef
Atsushi Masumori
6577ce7f9c5c26f96ce634f0
Takashi Ikegami
From Text to Motion: Grounding GPT-4 in a Humanoid Robot "Alter3"
We report the development of Alter3, a humanoid robot capable of generating
spontaneous motion using a Large Language Model (LLM), specifically GPT-4. This
achievement was realized by integrating GPT-4 into our proprietary android,
Alter3, thereby effectively grounding the LLM with Alter's bodily movement.
Typically, low-level robot control is hardware-dependent and falls outside the
scope of LLM corpora, presenting challenges for direct LLM-based robot control.
However, in the case of humanoid robots like Alter3, direct control is feasible
by mapping the linguistic expressions of human actions onto the robot's body
through program code. Remarkably, this approach enables Alter3 to adopt various
poses, such as a 'selfie' stance or 'pretending to be a ghost,' and generate
sequences of actions over time without explicit programming for each body part.
This demonstrates the robot's zero-shot learning capabilities. Additionally,
verbal feedback can adjust poses, obviating the need for fine-tuning. A video
of Alter3's generated motions is available at
https://tnoinkwms.github.io/ALTER-LLM/
A humanoid robot, Alter3, uses a Large Language Model to generate spontaneous and varied body movements, demonstrating zero-shot learning and adjustability through verbal feedback.
Large Language Model LLM GPT-4 humanoid robot zero-shot learning