Segment Any Motion in Videos
Moving object segmentation is a crucial task for achieving a high-level
understanding of visual scenes and has numerous downstream applications. Humans
can effortlessly segment moving objects in videos. Previous work has largely
relied on optical flow to provide motion cues; however, this approach often
results in imperfect predictions due to challenges such as partial motion,
complex deformations, motion blur and background distractions. We propose a
novel approach for moving object segmentation that combines long-range
trajectory motion cues with DINO-based semantic features and leverages SAM2 for
pixel-level mask densification through an iterative prompting strategy. Our
model employs Spatio-Temporal Trajectory Attention and Motion-Semantic
Decoupled Embedding to prioritize motion while integrating semantic support.
Extensive testing on diverse datasets demonstrates state-of-the-art
performance, excelling in challenging scenarios and fine-grained segmentation
of multiple objects. Our code is available at https://motion-seg.github.io/.
67e9fec1564b123aa5d70388
Nan Huang
67e9fec1564b123aa5d70389
Wenzhao Zheng
67e9fec1564b123aa5d7038a
Chenfeng Xu
admin_assigned
2025-03-31T09:56:25.025Z
654ca71d5255ee86711b52c5
/avatars/52bf00fd74c8db5643c4daa185c678e6.svg
Chenfeng Xu
user
chenfengx
67e9fec1564b123aa5d7038b
Kurt Keutzer
admin_assigned
2025-03-31T09:56:18.832Z
6251bf4b183aa4266924ad91
https://cdn-avatars.huggingface.co/v1/production/uploads/1678041834400-6251bf4b183aa4266924ad91.jpeg
Kurt Keutzer
user
kurtkeutzer
67e9fec1564b123aa5d7038c
Shanghang Zhang
67e9fec1564b123aa5d7038d
Angjoo Kanazawa
admin_assigned
2025-03-31T09:56:03.262Z
6478cf7150ff7001631679c3
/avatars/65ec385a9cc44c972e6caf952e759ff1.svg
Angjoo Kanazawa
user
akanazawa
67e9fec1564b123aa5d7038e
Qianqian Wang
admin_assigned
2025-03-31T09:55:54.563Z
6616f0c4c2e30710e607c2bf
/avatars/a5941e0ed940439f4c7c67747318cbfc.svg
Qianqian Wang
user
qianqian68
Segment Any Motion in Videos
Moving object segmentation is a crucial task for achieving a high-level
understanding of visual scenes and has numerous downstream applications. Humans
can effortlessly segment moving objects in videos. Previous work has largely
relied on optical flow to provide motion cues; however, this approach often
results in imperfect predictions due to challenges such as partial motion,
complex deformations, motion blur and background distractions. We propose a
novel approach for moving object segmentation that combines long-range
trajectory motion cues with DINO-based semantic features and leverages SAM2 for
pixel-level mask densification through an iterative prompting strategy. Our
model employs Spatio-Temporal Trajectory Attention and Motion-Semantic
Decoupled Embedding to prioritize motion while integrating semantic support.
Extensive testing on diverse datasets demonstrates state-of-the-art
performance, excelling in challenging scenarios and fine-grained segmentation
of multiple objects. Our code is available at https://motion-seg.github.io/.
A novel approach combining long-range trajectory motion cues with DINO-based semantic features and SAM2 for moving object segmentation achieves state-of-the-art performance in challenging scenarios.
DINO SAM2 Spatio-Temporal Trajectory Attention Motion-Semantic Decoupled Embedding