Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual
  Tokens
Vision-language models (VLMs) excel at multimodal understanding, yet their
text-only decoding forces them to verbalize visual reasoning, limiting
performance on tasks that demand visual imagination. Recent attempts train VLMs
to render explicit images, but the heavy image-generation pre-training often
hinders the reasoning ability. Inspired by the way humans reason with mental
imagery-the internal construction and manipulation of visual cues-we
investigate whether VLMs can reason through interleaved multimodal trajectories
without producing explicit images. To this end, we present a Machine Mental
Imagery framework, dubbed as Mirage, which augments VLM decoding with latent
visual tokens alongside ordinary text. Concretely, whenever the model chooses
to ``think visually'', it recasts its hidden states as next tokens, thereby
continuing a multimodal trajectory without generating pixel-level images. Begin
by supervising the latent tokens through distillation from ground-truth image
embeddings, we then switch to text-only supervision to make the latent
trajectory align tightly with the task objective. A subsequent reinforcement
learning stage further enhances the multimodal reasoning capability.
Experiments on diverse benchmarks demonstrate that Mirage unlocks stronger
multimodal reasoning without explicit image generation.
68592f3b0e4ad7e219758344
Zeyuan Yang
claimed_verified
2025-06-27T20:31:14.627Z
6483065104f67f5f6060cba2
/avatars/e1bbcf9d7cc0d438d9bdc25f340fa0ac.svg
Zeyuan Yang
user
Miiche
68592f3b0e4ad7e219758345
Xueyang Yu
claimed_verified
2025-06-24T08:09:21.848Z
643340516c2a26ae66d5baa6
/avatars/bcf5aba7d5778160aaf5080155eafe7a.svg
Xueyang Yu
user
XueyangY
68592f3b0e4ad7e219758346
Delin Chen
68592f3b0e4ad7e219758347
Maohao Shen
68592f3b0e4ad7e219758348
Chuang Gan
Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual
  Tokens
Vision-language models (VLMs) excel at multimodal understanding, yet their
text-only decoding forces them to verbalize visual reasoning, limiting
performance on tasks that demand visual imagination. Recent attempts train VLMs
to render explicit images, but the heavy image-generation pre-training often
hinders the reasoning ability. Inspired by the way humans reason with mental
imagery-the internal construction and manipulation of visual cues-we
investigate whether VLMs can reason through interleaved multimodal trajectories
without producing explicit images. To this end, we present a Machine Mental
Imagery framework, dubbed as Mirage, which augments VLM decoding with latent
visual tokens alongside ordinary text. Concretely, whenever the model chooses
to ``think visually'', it recasts its hidden states as next tokens, thereby
continuing a multimodal trajectory without generating pixel-level images. Begin
by supervising the latent tokens through distillation from ground-truth image
embeddings, we then switch to text-only supervision to make the latent
trajectory align tightly with the task objective. A subsequent reinforcement
learning stage further enhances the multimodal reasoning capability.
Experiments on diverse benchmarks demonstrate that Mirage unlocks stronger
multimodal reasoning without explicit image generation.
Mirage enhances vision-language models by integrating latent visual tokens into text decoding to improve multimodal reasoning without generating explicit images.
vision-language models text-only decoding visual imagination image-generation pre-training latent visual tokens multimodal trajectories hidden states distillation ground-truth image embeddings reinforcement learning multimodal reasoning