ERGO: Efficient High-Resolution Visual Understanding for Vision-Language
  Models
Efficient processing of high-resolution images is crucial for real-world
vision-language applications. However, existing Large Vision-Language Models
(LVLMs) incur substantial computational overhead due to the large number of
vision tokens. With the advent of "thinking with images" models, reasoning now
extends beyond text to the visual domain. This capability motivates our
two-stage "coarse-to-fine" reasoning pipeline: first, a downsampled image is
analyzed to identify task-relevant regions; then, only these regions are
cropped at full resolution and processed in a subsequent reasoning stage. This
approach reduces computational cost while preserving fine-grained visual
details where necessary. A major challenge lies in inferring which regions are
truly relevant to a given query. Recent related methods often fail in the first
stage after input-image downsampling, due to perception-driven reasoning, where
clear visual information is required for effective reasoning. To address this
issue, we propose ERGO (Efficient Reasoning & Guided Observation) that performs
reasoning-driven perception-leveraging multimodal context to determine where to
focus. Our model can account for perceptual uncertainty, expanding the cropped
region to cover visually ambiguous areas for answering questions. To this end,
we develop simple yet effective reward components in a reinforcement learning
framework for coarse-to-fine perception. Across multiple datasets, our approach
delivers higher accuracy than the original model and competitive methods, with
greater efficiency. For instance, ERGO surpasses Qwen2.5-VL-7B on the V*
benchmark by 4.7 points while using only 23% of the vision tokens, achieving a
3x inference speedup. The code and models can be found at:
https://github.com/nota-github/ERGO.
68da629acfa03a5158a1ed9c
Jewon Lee
claimed_verified
2025-09-29T13:05:11.586Z
642d395784bf892b8fa91816
/avatars/2db2202641ad7ed9aeaea7e8bfc3acca.svg
Jewon Lee
user
je1lee
68da629acfa03a5158a1ed9d
Wooksu Shin
claimed_verified
2025-09-29T13:05:14.261Z
67a0a0d48f047b67c3172e9e
/avatars/fa2ee986071f3642c8c7cd6ff693dbfd.svg
Wooksu Shin
user
dnrtn1101
68da629acfa03a5158a1ed9e
Seungmin Yang
68da629acfa03a5158a1ed9f
Ki-Ung Song
68da629acfa03a5158a1eda0
DongUk Lim
68da629acfa03a5158a1eda1
Jaeyeon Kim
68da629acfa03a5158a1eda2
Tae-Ho Kim
68da629acfa03a5158a1eda3
Bo-Kyeong Kim
claimed_verified
2025-09-29T11:28:06.616Z
64374f10cd93f4c9a34c1239
https://cdn-avatars.huggingface.co/v1/production/uploads/64374f10cd93f4c9a34c1239/tdkdcgl18rKXLvyTEhfnG.jpeg
Bo-Kyeong Kim
user
bokyeong1015
ERGO: Efficient High-Resolution Visual Understanding for Vision-Language
  Models
Efficient processing of high-resolution images is crucial for real-world
vision-language applications. However, existing Large Vision-Language Models
(LVLMs) incur substantial computational overhead due to the large number of
vision tokens. With the advent of "thinking with images" models, reasoning now
extends beyond text to the visual domain. This capability motivates our
two-stage "coarse-to-fine" reasoning pipeline: first, a downsampled image is
analyzed to identify task-relevant regions; then, only these regions are
cropped at full resolution and processed in a subsequent reasoning stage. This
approach reduces computational cost while preserving fine-grained visual
details where necessary. A major challenge lies in inferring which regions are
truly relevant to a given query. Recent related methods often fail in the first
stage after input-image downsampling, due to perception-driven reasoning, where
clear visual information is required for effective reasoning. To address this
issue, we propose ERGO (Efficient Reasoning & Guided Observation) that performs
reasoning-driven perception-leveraging multimodal context to determine where to
focus. Our model can account for perceptual uncertainty, expanding the cropped
region to cover visually ambiguous areas for answering questions. To this end,
we develop simple yet effective reward components in a reinforcement learning
framework for coarse-to-fine perception. Across multiple datasets, our approach
delivers higher accuracy than the original model and competitive methods, with
greater efficiency. For instance, ERGO surpasses Qwen2.5-VL-7B on the V*
benchmark by 4.7 points while using only 23% of the vision tokens, achieving a
3x inference speedup. The code and models can be found at:
https://github.com/nota-github/ERGO.
ERGO, a two-stage reasoning pipeline, efficiently processes high-resolution images by focusing on task-relevant regions, improving accuracy and speed compared to existing models.
Large Vision-Language Models LVLMs coarse-to-fine reasoning downsampled image full resolution reasoning-driven perception multimodal context perceptual uncertainty reinforcement learning V* benchmark vision tokens inference speedup