Agile Catching with Whole-Body MPC and Blackbox Policy Learning
We address a benchmark task in agile robotics: catching objects thrown at
high-speed. This is a challenging task that involves tracking, intercepting,
and cradling a thrown object with access only to visual observations of the
object and the proprioceptive state of the robot, all within a fraction of a
second. We present the relative merits of two fundamentally different solution
strategies: (i) Model Predictive Control using accelerated constrained
trajectory optimization, and (ii) Reinforcement Learning using zeroth-order
optimization. We provide insights into various performance trade-offs including
sample efficiency, sim-to-real transfer, robustness to distribution shifts, and
whole-body multimodality via extensive on-hardware experiments. We conclude
with proposals on fusing "classical" and "learning-based" techniques for agile
robot control. Videos of our experiments may be found at
https://sites.google.com/view/agile-catching
648c028817c7ceb9b41a40a1
Saminda Abeyruwan
648c028817c7ceb9b41a40a2
Alex Bewley
claimed_verified
2023-06-21T20:46:05.285Z
6492dfe783ff9ba7a9bbdeea
/avatars/e1d8be8bb68200230cdf05497479e6f4.svg
Alex Bewley
user
bewley
648c028817c7ceb9b41a40a3
Nicholas M. Boffi
claimed_verified
2024-01-23T17:09:35.679Z
65afd8b7bd418aab3a677782
/avatars/11aa33b32ef4b6aa27ab8e6c068afa14.svg
Nicholas Boffi
user
nmboffi
648c028817c7ceb9b41a40a4
Krzysztof Choromanski
648c028817c7ceb9b41a40a5
David D'Ambrosio
648c028817c7ceb9b41a40a6
Deepali Jain
claimed_verified
2023-06-16T17:38:19.893Z
648c741b39d2584ee46ebc7b
/avatars/ad3e078c6df8b319ef5fa85e556f8a3c.svg
Deepali Jain
user
jaindeepali
648c028817c7ceb9b41a40a7
Pannag Sanketi
claimed_verified
2024-08-09T07:49:26.766Z
66b56f73fc66a466e6104bfb
/avatars/74855aaf580144fafead4b93d6a24a46.svg
Pannag Sanketi
user
prsanky
648c028817c7ceb9b41a40a8
Anish Shankar
648c028817c7ceb9b41a40a9
Vikas Sindhwani
648c028817c7ceb9b41a40aa
Sumeet Singh
648c028817c7ceb9b41a40ab
Jean-Jacques Slotine
648c028817c7ceb9b41a40ac
Stephen Tu
Agile Catching with Whole-Body MPC and Blackbox Policy Learning
We address a benchmark task in agile robotics: catching objects thrown at
high-speed. This is a challenging task that involves tracking, intercepting,
and cradling a thrown object with access only to visual observations of the
object and the proprioceptive state of the robot, all within a fraction of a
second. We present the relative merits of two fundamentally different solution
strategies: (i) Model Predictive Control using accelerated constrained
trajectory optimization, and (ii) Reinforcement Learning using zeroth-order
optimization. We provide insights into various performance trade-offs including
sample efficiency, sim-to-real transfer, robustness to distribution shifts, and
whole-body multimodality via extensive on-hardware experiments. We conclude
with proposals on fusing "classical" and "learning-based" techniques for agile
robot control. Videos of our experiments may be found at
https://sites.google.com/view/agile-catching
Two methodologies, Model Predictive Control with constrained trajectory optimization and Reinforcement Learning with zeroth-order optimization, are compared for high-speed object catching in robotics, highlighting performance metrics and potential integration strategies.
Model Predictive Control constrained trajectory optimization Reinforcement Learning zeroth-order optimization proprioceptive state sim-to-real transfer distribution shifts whole-body multimodality