DragAPart: Learning a Part-Level Motion Prior for Articulated Objects
We introduce DragAPart, a method that, given an image and a set of drags as
input, can generate a new image of the same object in a new state, compatible
with the action of the drags. Differently from prior works that focused on
repositioning objects, DragAPart predicts part-level interactions, such as
opening and closing a drawer. We study this problem as a proxy for learning a
generalist motion model, not restricted to a specific kinematic structure or
object category. To this end, we start from a pre-trained image generator and
fine-tune it on a new synthetic dataset, Drag-a-Move, which we introduce.
Combined with a new encoding for the drags and dataset randomization, the new
model generalizes well to real images and different categories. Compared to
prior motion-controlled generators, we demonstrate much better part-level
motion understanding.
660108c1e73568bf2ee5a8aa
Ruining Li
claimed_verified
2024-03-26T07:54:26.576Z
6467cc17e7a6a374fd1a41e5
/avatars/3cbd4ad5bc17067ee38b13d0cbadd80c.svg
Ruining Li
user
rayli
660108c1e73568bf2ee5a8ab
Chuanxia Zheng
claimed_verified
2024-03-25T11:30:09.059Z
635e16911e2d8c6673e39785
/avatars/905bccbe96b72862515ae4c27e7de642.svg
chuanxia
user
lyndonzheng
660108c1e73568bf2ee5a8ac
Christian Rupprecht
660108c1e73568bf2ee5a8ad
Andrea Vedaldi
DragAPart: Learning a Part-Level Motion Prior for Articulated Objects
We introduce DragAPart, a method that, given an image and a set of drags as
input, can generate a new image of the same object in a new state, compatible
with the action of the drags. Differently from prior works that focused on
repositioning objects, DragAPart predicts part-level interactions, such as
opening and closing a drawer. We study this problem as a proxy for learning a
generalist motion model, not restricted to a specific kinematic structure or
object category. To this end, we start from a pre-trained image generator and
fine-tune it on a new synthetic dataset, Drag-a-Move, which we introduce.
Combined with a new encoding for the drags and dataset randomization, the new
model generalizes well to real images and different categories. Compared to
prior motion-controlled generators, we demonstrate much better part-level
motion understanding.
DragAPart generates new images based on part-level interactions using fine-tuned image generators and a new synthetic dataset, showing improved part-level motion understanding compared to previous methods.
DragAPart image generator fine-tuning Drag-a-Move part-level interactions