Scaling Inference-Time Search with Vision Value Model for Improved
  Visual Comprehension
Despite significant advancements in vision-language models (VLMs), there
lacks effective approaches to enhance response quality by scaling
inference-time computation. This capability is known to be a core step towards
the self-improving models in recent large language model studies. In this
paper, we present Vision Value Model (VisVM) that can guide VLM inference-time
search to generate responses with better visual comprehension. Specifically,
VisVM not only evaluates the generated sentence quality in the current search
step, but also anticipates the quality of subsequent sentences that may result
from the current step, thus providing a long-term value. In this way, VisVM
steers VLMs away from generating sentences prone to hallucinations or
insufficient detail, thereby producing higher quality responses. Experimental
results demonstrate that VisVM-guided search significantly enhances VLMs'
ability to generate descriptive captions with richer visual details and fewer
hallucinations, compared with greedy decoding and search methods with other
visual reward signals. Furthermore, we find that self-training the model with
the VisVM-guided captions improve VLM's performance across a wide range of
multimodal benchmarks, indicating the potential for developing self-improving
VLMs. Our value model and code are available at
https://github.com/si0wang/VisVM.
675320b0cb11c5f0308bf62b
Wang Xiyao
claimed_verified
2024-12-06T17:14:30.976Z
655fed9fdef5905d38b84af3
/avatars/2cda4182dfd11a1e94743639e62328ea.svg
Xiyao Wang
user
russwang
675320b0cb11c5f0308bf62c
Yang Zhengyuan
675320b0cb11c5f0308bf62d
Li Linjie
675320b0cb11c5f0308bf62e
Lu Hongjin
675320b0cb11c5f0308bf62f
Xu Yuancheng
675320b0cb11c5f0308bf630
Lin Chung-Ching Lin
675320b0cb11c5f0308bf631
Lin Kevin
675320b0cb11c5f0308bf632
Huang Furong
675320b0cb11c5f0308bf633
Wang Lijuan
Scaling Inference-Time Search with Vision Value Model for Improved
  Visual Comprehension
Despite significant advancements in vision-language models (VLMs), there
lacks effective approaches to enhance response quality by scaling
inference-time computation. This capability is known to be a core step towards
the self-improving models in recent large language model studies. In this
paper, we present Vision Value Model (VisVM) that can guide VLM inference-time
search to generate responses with better visual comprehension. Specifically,
VisVM not only evaluates the generated sentence quality in the current search
step, but also anticipates the quality of subsequent sentences that may result
from the current step, thus providing a long-term value. In this way, VisVM
steers VLMs away from generating sentences prone to hallucinations or
insufficient detail, thereby producing higher quality responses. Experimental
results demonstrate that VisVM-guided search significantly enhances VLMs'
ability to generate descriptive captions with richer visual details and fewer
hallucinations, compared with greedy decoding and search methods with other
visual reward signals. Furthermore, we find that self-training the model with
the VisVM-guided captions improve VLM's performance across a wide range of
multimodal benchmarks, indicating the potential for developing self-improving
VLMs. Our value model and code are available at
https://github.com/si0wang/VisVM.
VisVM enhances vision-language model inference by guiding sentence generation to improve visual comprehension and reduce hallucinations.
Vision Value Model VLMs inference-time search generated sentence quality visual comprehension hallucinations visual reward signals multimodal benchmarks self-improving VLMs