Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified
  Self-Play
Although reinforcement learning (RL) can effectively enhance the reasoning
capabilities of vision-language models (VLMs), current methods remain heavily
dependent on labor-intensive datasets that require extensive manual
construction and verification, leading to extremely high training costs and
consequently constraining the practical deployment of VLMs. To address this
challenge, we propose Vision-Zero, a domain-agnostic framework enabling VLM
self-improvement through competitive visual games generated from arbitrary
image pairs. Specifically, Vision-Zero encompasses three main attributes: (1)
Strategic Self-Play Framework: Vision-Zero trains VLMs in "Who Is the
Spy"-style games, where the models engage in strategic reasoning and actions
across multiple roles. Through interactive gameplay, models autonomously
generate their training data without human annotation. (2) Gameplay from
Arbitrary Images: Unlike existing gamified frameworks, Vision-Zero can generate
games from arbitrary images, thereby enhancing the model's reasoning ability
across diverse domains and showing strong generalization to different tasks. We
demonstrate this versatility using three distinct types of image datasets:
CLEVR-based synthetic scenes, charts, and real-world images. (3) Sustainable
Performance Gain: We introduce Iterative Self-Play Policy Optimization
(Iterative-SPO), a novel training algorithm that alternates between Self-Play
and reinforcement learning with verifiable rewards (RLVR), mitigating the
performance plateau often seen in self-play-only training and achieving
sustained long-term improvements. Despite using label-free data, Vision-Zero
achieves state-of-the-art performance on reasoning, chart question answering,
and vision-centric understanding tasks, surpassing other annotation-based
methods. Models and code has been released at
https://github.com/wangqinsi1/Vision-Zero.
68dc90f34159d1f2418f9abf
Qinsi Wang
68dc90f34159d1f2418f9ac0
Bo Liu
claimed_verified
2025-10-01T10:21:49.246Z
635e3a76106f984574c36409
https://cdn-avatars.huggingface.co/v1/production/uploads/1667120725800-635e3a76106f984574c36409.png
Bo Liu
user
Benjamin-eecs
68dc90f34159d1f2418f9ac1
Tianyi Zhou
claimed_verified
2025-10-01T10:21:44.939Z
647f5af5b0e96764589f3b2a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
Tianyi Zhou
user
zhoutianyi
68dc90f34159d1f2418f9ac2
Jing Shi
68dc90f34159d1f2418f9ac3
Yueqian Lin
claimed_verified
2025-10-02T03:09:00.258Z
64b5198c25882acb62fb77ef
https://cdn-avatars.huggingface.co/v1/production/uploads/64b5198c25882acb62fb77ef/HX9pfMEPQlfjvSAgSLplY.png
Yueqian Lin
user
linyueqian
68dc90f34159d1f2418f9ac4
Yiran Chen
68dc90f34159d1f2418f9ac5
Hai Helen Li
68dc90f34159d1f2418f9ac6
Kun Wan
claimed_verified
2025-10-01T10:21:47.064Z
66274e02348a5304435dc9cc
/avatars/bda87559cd497c310597c2fc8430b31f.svg
Kun Wan
user
timecuriosity
68dc90f34159d1f2418f9ac7
Wentian Zhao
Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified
  Self-Play
Although reinforcement learning (RL) can effectively enhance the reasoning
capabilities of vision-language models (VLMs), current methods remain heavily
dependent on labor-intensive datasets that require extensive manual
construction and verification, leading to extremely high training costs and
consequently constraining the practical deployment of VLMs. To address this
challenge, we propose Vision-Zero, a domain-agnostic framework enabling VLM
self-improvement through competitive visual games generated from arbitrary
image pairs. Specifically, Vision-Zero encompasses three main attributes: (1)
Strategic Self-Play Framework: Vision-Zero trains VLMs in "Who Is the
Spy"-style games, where the models engage in strategic reasoning and actions
across multiple roles. Through interactive gameplay, models autonomously
generate their training data without human annotation. (2) Gameplay from
Arbitrary Images: Unlike existing gamified frameworks, Vision-Zero can generate
games from arbitrary images, thereby enhancing the model's reasoning ability
across diverse domains and showing strong generalization to different tasks. We
demonstrate this versatility using three distinct types of image datasets:
CLEVR-based synthetic scenes, charts, and real-world images. (3) Sustainable
Performance Gain: We introduce Iterative Self-Play Policy Optimization
(Iterative-SPO), a novel training algorithm that alternates between Self-Play
and reinforcement learning with verifiable rewards (RLVR), mitigating the
performance plateau often seen in self-play-only training and achieving
sustained long-term improvements. Despite using label-free data, Vision-Zero
achieves state-of-the-art performance on reasoning, chart question answering,
and vision-centric understanding tasks, surpassing other annotation-based
methods. Models and code has been released at
https://github.com/wangqinsi1/Vision-Zero.
Vision-Zero is a domain-agnostic framework that enhances vision-language models through self-improvement in competitive visual games, using Iterative Self-Play Policy Optimization and achieving state-of-the-art performance without human annotation.
reinforcement learning vision-language models strategic self-play framework self-play reinforcement learning with verifiable rewards Iterative Self-Play Policy Optimization