Dynamic-Resolution Model Learning for Object Pile Manipulation
Dynamics models learned from visual observations have shown to be effective
in various robotic manipulation tasks. One of the key questions for learning
such dynamics models is what scene representation to use. Prior works typically
assume representation at a fixed dimension or resolution, which may be
inefficient for simple tasks and ineffective for more complicated tasks. In
this work, we investigate how to learn dynamic and adaptive representations at
different levels of abstraction to achieve the optimal trade-off between
efficiency and effectiveness. Specifically, we construct dynamic-resolution
particle representations of the environment and learn a unified dynamics model
using graph neural networks (GNNs) that allows continuous selection of the
abstraction level. During test time, the agent can adaptively determine the
optimal resolution at each model-predictive control (MPC) step. We evaluate our
method in object pile manipulation, a task we commonly encounter in cooking,
agriculture, manufacturing, and pharmaceutical applications. Through
comprehensive evaluations both in the simulation and the real world, we show
that our method achieves significantly better performance than state-of-the-art
fixed-resolution baselines at the gathering, sorting, and redistribution of
granular object piles made with various instances like coffee beans, almonds,
corn, etc.
649e53f81b00eaf1eeb03faf
Yixuan Wang
claimed_verified
2023-07-03T07:27:19.571Z
649f4eca850cf879852b6cbc
/avatars/1253925dce9875eaa54332be59149a1b.svg
Yixuan Wang
user
yixuan1999
649e53f81b00eaf1eeb03fb0
Yunzhu Li
claimed_verified
2023-07-04T19:17:25.073Z
64a46562c641afb468a3dc04
/avatars/bc54e5c13f657251b05bf2725086e56d.svg
Yunzhu Li
user
yunzhuli
649e53f81b00eaf1eeb03fb1
Katherine Driggs-Campbell
649e53f81b00eaf1eeb03fb2
Li Fei-Fei
649e53f81b00eaf1eeb03fb3
Jiajun Wu
claimed_verified
2023-06-30T20:55:55.157Z
647b7b8345616c3de3627863
/avatars/8196364bffed71086ba9a0a4a62027d3.svg
Jiajun Wu
user
jiajunwu
Dynamic-Resolution Model Learning for Object Pile Manipulation
Dynamics models learned from visual observations have shown to be effective
in various robotic manipulation tasks. One of the key questions for learning
such dynamics models is what scene representation to use. Prior works typically
assume representation at a fixed dimension or resolution, which may be
inefficient for simple tasks and ineffective for more complicated tasks. In
this work, we investigate how to learn dynamic and adaptive representations at
different levels of abstraction to achieve the optimal trade-off between
efficiency and effectiveness. Specifically, we construct dynamic-resolution
particle representations of the environment and learn a unified dynamics model
using graph neural networks (GNNs) that allows continuous selection of the
abstraction level. During test time, the agent can adaptively determine the
optimal resolution at each model-predictive control (MPC) step. We evaluate our
method in object pile manipulation, a task we commonly encounter in cooking,
agriculture, manufacturing, and pharmaceutical applications. Through
comprehensive evaluations both in the simulation and the real world, we show
that our method achieves significantly better performance than state-of-the-art
fixed-resolution baselines at the gathering, sorting, and redistribution of
granular object piles made with various instances like coffee beans, almonds,
corn, etc.
A dynamic-resolution particle representation using graph neural networks achieves better performance in object pile manipulation tasks compared to fixed-resolution models.
graph neural networks dynamic-resolution particle representations model-predictive control object pile manipulation