BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human
  Racing Gameplay
Imitation learning learns a policy from demonstrations without requiring
hand-designed reward functions. In many robotic tasks, such as autonomous
racing, imitated policies must model complex environment dynamics and human
decision-making. Sequence modeling is highly effective in capturing intricate
patterns of motion sequences but struggles to adapt to new environments or
distribution shifts that are common in real-world robotics tasks. In contrast,
Adversarial Imitation Learning (AIL) can mitigate this effect, but struggles
with sample inefficiency and handling complex motion patterns. Thus, we propose
BeTAIL: Behavior Transformer Adversarial Imitation Learning, which combines a
Behavior Transformer (BeT) policy from human demonstrations with online AIL.
BeTAIL adds an AIL residual policy to the BeT policy to model the sequential
decision-making process of human experts and correct for out-of-distribution
states or shifts in environment dynamics. We test BeTAIL on three challenges
with expert-level demonstrations of real human gameplay in Gran Turismo Sport.
Our proposed residual BeTAIL reduces environment interactions and improves
racing performance and stability, even when the BeT is pretrained on different
tracks than downstream learning. Videos and code available at:
https://sites.google.com/berkeley.edu/BeTAIL/home.
65d8224b984cc240f21e0743
Catherine Weaver
65d8224b984cc240f21e0744
Chen Tang
65d8224b984cc240f21e0745
Ce Hao
65d8224b984cc240f21e0746
Kenta Kawamoto
65d8224b984cc240f21e0747
Masayoshi Tomizuka
65d8224b984cc240f21e0748
Wei Zhan
BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human
  Racing Gameplay
Imitation learning learns a policy from demonstrations without requiring
hand-designed reward functions. In many robotic tasks, such as autonomous
racing, imitated policies must model complex environment dynamics and human
decision-making. Sequence modeling is highly effective in capturing intricate
patterns of motion sequences but struggles to adapt to new environments or
distribution shifts that are common in real-world robotics tasks. In contrast,
Adversarial Imitation Learning (AIL) can mitigate this effect, but struggles
with sample inefficiency and handling complex motion patterns. Thus, we propose
BeTAIL: Behavior Transformer Adversarial Imitation Learning, which combines a
Behavior Transformer (BeT) policy from human demonstrations with online AIL.
BeTAIL adds an AIL residual policy to the BeT policy to model the sequential
decision-making process of human experts and correct for out-of-distribution
states or shifts in environment dynamics. We test BeTAIL on three challenges
with expert-level demonstrations of real human gameplay in Gran Turismo Sport.
Our proposed residual BeTAIL reduces environment interactions and improves
racing performance and stability, even when the BeT is pretrained on different
tracks than downstream learning. Videos and code available at:
https://sites.google.com/berkeley.edu/BeTAIL/home.
BeTAIL combines Behavior Transformer with Adversarial Imitation Learning to enhance robotic imitation in dynamic environments, improving performance and stability with sample efficiency.
imitation learning Adversarial Imitation Learning (AIL) Behavior Transformer (BeT) residual policy sequential decision-making human experts environment dynamics distribution shifts real-world robotics tasks expert-level demonstrations Gran Turismo Sport