FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid
  Control
Reinforcement learning (RL) has driven significant progress in robotics, but
its complexity and long training times remain major bottlenecks. In this
report, we introduce FastTD3, a simple, fast, and capable RL algorithm that
significantly speeds up training for humanoid robots in popular suites such as
HumanoidBench, IsaacLab, and MuJoCo Playground. Our recipe is remarkably
simple: we train an off-policy TD3 agent with several modifications -- parallel
simulation, large-batch updates, a distributional critic, and carefully tuned
hyperparameters. FastTD3 solves a range of HumanoidBench tasks in under 3 hours
on a single A100 GPU, while remaining stable during training. We also provide a
lightweight and easy-to-use implementation of FastTD3 to accelerate RL research
in robotics.
6838a05e80df874a4f2360d8
Younggyo Seo
6838a05e80df874a4f2360d9
Carmelo Sferrazza
6838a05e80df874a4f2360da
Haoran Geng
6838a05e80df874a4f2360db
Michal Nauman
6838a05e80df874a4f2360dc
Zhao-Heng Yin
6838a05e80df874a4f2360dd
Pieter Abbeel
FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid
  Control
Reinforcement learning (RL) has driven significant progress in robotics, but
its complexity and long training times remain major bottlenecks. In this
report, we introduce FastTD3, a simple, fast, and capable RL algorithm that
significantly speeds up training for humanoid robots in popular suites such as
HumanoidBench, IsaacLab, and MuJoCo Playground. Our recipe is remarkably
simple: we train an off-policy TD3 agent with several modifications -- parallel
simulation, large-batch updates, a distributional critic, and carefully tuned
hyperparameters. FastTD3 solves a range of HumanoidBench tasks in under 3 hours
on a single A100 GPU, while remaining stable during training. We also provide a
lightweight and easy-to-use implementation of FastTD3 to accelerate RL research
in robotics.
FastTD3, an enhanced RL algorithm with parallel simulation and distributional critic, significantly accelerates training for humanoid robots.
reinforcement learning RL FastTD3 off-policy TD3 parallel simulation large-batch updates distributional critic HumanoidBench IsaacLab MuJoCo Playground