MotionSight: Boosting Fine-Grained Motion Understanding in Multimodal
  LLMs
Despite advancements in Multimodal Large Language Models (MLLMs), their
proficiency in fine-grained video motion understanding remains critically
limited. They often lack inter-frame differencing and tend to average or ignore
subtle visual cues. Furthermore, while visual prompting has shown potential in
static images, its application to video's temporal complexities, particularly
for fine-grained motion understanding, remains largely unexplored. We
investigate whether inherent capability can be unlocked and boost MLLMs' motion
perception and enable distinct visual signatures tailored to decouple object
and camera motion cues. In this study, we introduce MotionSight, a novel
zero-shot method pioneering object-centric visual spotlight and motion blur as
visual prompts to effectively improve fine-grained motion understanding without
training. To convert this into valuable data assets, we curated MotionVid-QA,
the first large-scale dataset for fine-grained video motion understanding, with
hierarchical annotations including SFT and preference data, {\Theta}(40K) video
clips and {\Theta}(87K) QAs. Experiments show MotionSight achieves
state-of-the-art open-source performance and competitiveness with commercial
models. In particular, for fine-grained motion understanding we present a novel
zero-shot technique and a large-scale, high-quality dataset. All the code and
annotations will be publicly available.
683faa6515abeae85e13336b
Yipeng Du
683faa6515abeae85e13336c
Tiehan Fan
claimed_verified
2025-06-10T09:28:54.016Z
65927f3b754092f6b1e187a7
https://cdn-avatars.huggingface.co/v1/production/uploads/65927f3b754092f6b1e187a7/gUrNvIQHmsl1vLwSUxpmL.jpeg
tiehan fan
user
AnonMegumi
683faa6515abeae85e13336d
Kepan Nan
claimed_verified
2025-06-11T08:36:59.807Z
64fafab460017eeec978a2a1
/avatars/c156bda4e3e48b449a35f38a1f25c497.svg
nkp
user
nkp37
683faa6515abeae85e13336e
Rui Xie
683faa6515abeae85e13336f
Penghao Zhou
683faa6515abeae85e133370
Xiang Li
683faa6515abeae85e133371
Jian Yang
683faa6515abeae85e133372
Zhenheng Yang
683faa6515abeae85e133373
Ying Tai
claimed_verified
2025-06-04T08:56:59.253Z
65734004769f3ee9bde1af10
/avatars/d6310ed861972fd691687d8f47413f33.svg
Ying Tai
user
yingtai
MotionSight: Boosting Fine-Grained Motion Understanding in Multimodal
  LLMs
Despite advancements in Multimodal Large Language Models (MLLMs), their
proficiency in fine-grained video motion understanding remains critically
limited. They often lack inter-frame differencing and tend to average or ignore
subtle visual cues. Furthermore, while visual prompting has shown potential in
static images, its application to video's temporal complexities, particularly
for fine-grained motion understanding, remains largely unexplored. We
investigate whether inherent capability can be unlocked and boost MLLMs' motion
perception and enable distinct visual signatures tailored to decouple object
and camera motion cues. In this study, we introduce MotionSight, a novel
zero-shot method pioneering object-centric visual spotlight and motion blur as
visual prompts to effectively improve fine-grained motion understanding without
training. To convert this into valuable data assets, we curated MotionVid-QA,
the first large-scale dataset for fine-grained video motion understanding, with
hierarchical annotations including SFT and preference data, {\Theta}(40K) video
clips and {\Theta}(87K) QAs. Experiments show MotionSight achieves
state-of-the-art open-source performance and competitiveness with commercial
models. In particular, for fine-grained motion understanding we present a novel
zero-shot technique and a large-scale, high-quality dataset. All the code and
annotations will be publicly available.
MotionSight, a zero-shot method using object-centric visual spotlight and motion blur as prompts, enhances fine-grained video motion understanding and achieves state-of-the-art performance on MotionVid-QA, a large-scale dataset with hierarchical annotations.
Multimodal Large Language Models MLLMs fine-grained video motion understanding inter-frame differencing visual prompting temporal complexities MotionSight object-centric visual spotlight motion blur MotionVid-QA hierarchical annotations SFT preference data state-of-the-art performance