Visual Document Understanding and Question Answering: A Multi-Agent
  Collaboration Framework with Test-Time Scaling
Existing vision-language models (VLMs), whether generalists or specialists,
remain constrained by their parameter scale, lack robust self-correction
capabilities, and underperform in tasks involving long visual contexts and
complex reasoning, resulting in suboptimal performance on document-based tasks.
To address this, we propose MACT, a Multi-Agent Collaboration framework with
Test-Time scaling, tailored for visual document understanding and visual
question answering (VQA). It comprises four distinct small-scale agents, i.e.,
planning, execution, judgment, and answer agents, with clearly defined roles
and effective collaboration. Notably, the judgment agent exclusively verifies
correctness and redirects to prior agents for revisions, outperforming
conventional correction strategies. To further expand the capability boundaries
of the framework, we propose mixed reward modeling that balances agent-specific
abilities and global collaboration, as well as agent-wise hybrid test-time
scaling, which customizes different scaling strategies for each agent based on
their functions. Evaluated on benchmarks spanning both document-based and
non-document-based settings, our MACT shows superior performance with a smaller
parameter scale without sacrificing the ability of general and mathematical
tasks. Especially, it stands out in benchmarks involving long visual contexts
and complicated reasoning. The three variants of MACT consistently hold the top
three positions in average scores, leading in 13 of the 15 benchmarks. Code
will be available at: https://github.com/YU-deep/MACT.git.
68948599741a16f544fbd0a3
Xinlei Yu
68948599741a16f544fbd0a4
Zhangquan Chen
68948599741a16f544fbd0a5
Yudong Zhang
68948599741a16f544fbd0a6
Shilin Lu
claimed_verified
2025-09-26T12:24:23.439Z
631c4a23aa346997917bcb89
/avatars/4a562ba78e6be289049027c27798cc6e.svg
Shilin Lu
user
Shilin-LU
68948599741a16f544fbd0a7
Ruolin Shen
68948599741a16f544fbd0a8
Jiangning Zhang
68948599741a16f544fbd0a9
Xiaobin Hu
68948599741a16f544fbd0aa
Yanwei Fu
68948599741a16f544fbd0ab
Shuicheng Yan
Visual Document Understanding and Question Answering: A Multi-Agent
  Collaboration Framework with Test-Time Scaling
Existing vision-language models (VLMs), whether generalists or specialists,
remain constrained by their parameter scale, lack robust self-correction
capabilities, and underperform in tasks involving long visual contexts and
complex reasoning, resulting in suboptimal performance on document-based tasks.
To address this, we propose MACT, a Multi-Agent Collaboration framework with
Test-Time scaling, tailored for visual document understanding and visual
question answering (VQA). It comprises four distinct small-scale agents, i.e.,
planning, execution, judgment, and answer agents, with clearly defined roles
and effective collaboration. Notably, the judgment agent exclusively verifies
correctness and redirects to prior agents for revisions, outperforming
conventional correction strategies. To further expand the capability boundaries
of the framework, we propose mixed reward modeling that balances agent-specific
abilities and global collaboration, as well as agent-wise hybrid test-time
scaling, which customizes different scaling strategies for each agent based on
their functions. Evaluated on benchmarks spanning both document-based and
non-document-based settings, our MACT shows superior performance with a smaller
parameter scale without sacrificing the ability of general and mathematical
tasks. Especially, it stands out in benchmarks involving long visual contexts
and complicated reasoning. The three variants of MACT consistently hold the top
three positions in average scores, leading in 13 of the 15 benchmarks. Code
will be available at: https://github.com/YU-deep/MACT.git.
MACT, a Multi-Agent Collaboration framework with Test-Time scaling, enhances visual document understanding and VQA by using four specialized agents and mixed reward modeling, achieving superior performance with reduced parameters.
vision-language models VLMs Multi-Agent Collaboration Test-Time scaling visual document understanding visual question answering VQA planning agent execution agent judgment agent answer agent mixed reward modeling agent-wise hybrid test-time scaling long visual contexts complex reasoning