Grasp Any Region: Towards Precise, Contextual Pixel Understanding for
  Multimodal LLMs
While Multimodal Large Language Models (MLLMs) excel at holistic
understanding, they struggle in capturing the dense world with complex scenes,
requiring fine-grained analysis of intricate details and object
inter-relationships. Region-level MLLMs have been a promising step. However,
previous attempts are generally optimized to understand given regions in
isolation, neglecting crucial global contexts. To address this, we introduce
Grasp Any Region (GAR) for comprehen- sive region-level visual understanding.
Empowered by an effective RoI-aligned feature replay technique, GAR supports
(1) precise perception by leveraging necessary global contexts, and (2)
modeling interactions between multiple prompts. Together, it then naturally
achieves (3) advanced compositional reasoning to answer specific free-form
questions about any region, shifting the paradigm from passive description to
active dialogue. Moreover, we construct GAR-Bench, which not only provides a
more accurate evaluation of single-region comprehension, but also, more
importantly, measures interactions and complex reasoning across multiple
regions. Extensive experiments have demonstrated that GAR-1B not only maintains
the state-of-the-art captioning capabilities, e.g., outperforming DAM-3B +4.5
on DLC-Bench, but also excels at modeling relationships between multiple
prompts with advanced comprehension capabilities, even surpassing InternVL3-78B
on GAR-Bench-VQA. More importantly, our zero-shot GAR-8B even outperforms
in-domain VideoRefer-7B on VideoRefer-BenchQ, indicating its strong
capabilities can be easily transferred to videos.
68f837947669bcaeecce0be2
Haochen Wang
claimed_verified
2025-10-22T15:44:32.494Z
6499809cf19fc795e7724e43
/avatars/4b3adce8c85e2f3ef05318ded6c89c3e.svg
HaochenWang
user
HaochenWang
68f837947669bcaeecce0be3
Yuhao Wang
68f837947669bcaeecce0be4
Tao Zhang
68f837947669bcaeecce0be5
Yikang Zhou
68f837947669bcaeecce0be6
Yanwei Li
68f837947669bcaeecce0be7
Jiacong Wang
68f837947669bcaeecce0be8
Ye Tian
claimed_verified
2025-10-24T16:13:49.874Z
64e357dd825f4133e7427bf8
https://cdn-avatars.huggingface.co/v1/production/uploads/64e357dd825f4133e7427bf8/HwaWhINrzkbXG6SHG2oyf.jpeg
tyfeld
user
tyfeld
68f837947669bcaeecce0be9
Jiahao Meng
claimed_verified
2025-10-22T15:44:34.889Z
65a28e129acab19980226731
/avatars/abc3828f807efc4e03837b0eae063f98.svg
Jiahao Meng
user
marinero4972
68f837947669bcaeecce0bea
Zilong Huang
68f837947669bcaeecce0beb
Guangcan Mai
68f837947669bcaeecce0bec
Anran Wang
68f837947669bcaeecce0bed
Yunhai Tong
68f837947669bcaeecce0bee
Zhuochen Wang
68f837947669bcaeecce0bef
Xiangtai Li
68f837947669bcaeecce0bf0
Zhaoxiang Zhang
Grasp Any Region: Towards Precise, Contextual Pixel Understanding for
  Multimodal LLMs
While Multimodal Large Language Models (MLLMs) excel at holistic
understanding, they struggle in capturing the dense world with complex scenes,
requiring fine-grained analysis of intricate details and object
inter-relationships. Region-level MLLMs have been a promising step. However,
previous attempts are generally optimized to understand given regions in
isolation, neglecting crucial global contexts. To address this, we introduce
Grasp Any Region (GAR) for comprehen- sive region-level visual understanding.
Empowered by an effective RoI-aligned feature replay technique, GAR supports
(1) precise perception by leveraging necessary global contexts, and (2)
modeling interactions between multiple prompts. Together, it then naturally
achieves (3) advanced compositional reasoning to answer specific free-form
questions about any region, shifting the paradigm from passive description to
active dialogue. Moreover, we construct GAR-Bench, which not only provides a
more accurate evaluation of single-region comprehension, but also, more
importantly, measures interactions and complex reasoning across multiple
regions. Extensive experiments have demonstrated that GAR-1B not only maintains
the state-of-the-art captioning capabilities, e.g., outperforming DAM-3B +4.5
on DLC-Bench, but also excels at modeling relationships between multiple
prompts with advanced comprehension capabilities, even surpassing InternVL3-78B
on GAR-Bench-VQA. More importantly, our zero-shot GAR-8B even outperforms
in-domain VideoRefer-7B on VideoRefer-BenchQ, indicating its strong
capabilities can be easily transferred to videos.
Grasp Any Region (GAR) enhances region-level visual understanding by integrating global contexts and modeling interactions, achieving advanced reasoning and outperforming existing models in captioning and video reference tasks.
Multimodal Large Language Models Region-level MLLMs Grasp Any Region RoI-aligned feature replay compositional reasoning GAR-Bench captioning VideoRefer