Vision language models are blind
Large language models with vision capabilities (VLMs), e.g., GPT-4o and
Gemini 1.5 Pro are powering countless image-text applications and scoring high
on many vision-understanding benchmarks. Yet, we find that VLMs fail on 7
visual tasks absurdly easy to humans such as identifying (a) whether two
circles overlap; (b) whether two lines intersect; (c) which letter is being
circled in a word; and (d) counting the number of circles in a Olympic-like
logo. The shockingly poor performance of four state-of-the-art VLMs suggests
their vision is, at best, like of a person with myopia seeing fine details as
blurry, and at worst, like an intelligent person that is blind making educated
guesses. Code is available at: https://vlmsareblind.github.io/
668de4e71c1fe11f8c1d3c4a
Pooyan Rahmanzadehgervi
extracted_confirmed
2024-07-10T15:18:53.241Z
63e80a2f4577a86987a8ca8e
https://cdn-avatars.huggingface.co/v1/production/uploads/63e80a2f4577a86987a8ca8e/yB8QARfTb9OAa_pEICEbm.jpeg
Pooyan
user
pooyanrg
668de4e71c1fe11f8c1d3c4b
Logan Bolton
claimed_verified
2024-07-10T08:19:41.930Z
668c8e8c142f9b26a49f03cc
https://cdn-avatars.huggingface.co/v1/production/uploads/668c8e8c142f9b26a49f03cc/YNmPCrlsi6iwSeNfh1iID.png
Logan Bolton
user
loganbolton
668de4e71c1fe11f8c1d3c4c
Mohammad Reza Taesiri
extracted_confirmed
2024-07-10T01:38:10.673Z
6039478ab3ecf716b1a5fd4d
https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg
taesiri
user
taesiri
668de4e71c1fe11f8c1d3c4d
Anh Totti Nguyen
extracted_confirmed
2025-06-20T19:47:49.463Z
60e85b3fcd1cf4e418fff651
https://cdn-avatars.huggingface.co/v1/production/uploads/60e85b3fcd1cf4e418fff651/TUaEA4SqB6xuG2JhwNETC.jpeg
Anh (Totti) Nguyen
user
anhng8
Vision language models are blind
Large language models with vision capabilities (VLMs), e.g., GPT-4o and
Gemini 1.5 Pro are powering countless image-text applications and scoring high
on many vision-understanding benchmarks. Yet, we find that VLMs fail on 7
visual tasks absurdly easy to humans such as identifying (a) whether two
circles overlap; (b) whether two lines intersect; (c) which letter is being
circled in a word; and (d) counting the number of circles in a Olympic-like
logo. The shockingly poor performance of four state-of-the-art VLMs suggests
their vision is, at best, like of a person with myopia seeing fine details as
blurry, and at worst, like an intelligent person that is blind making educated
guesses. Code is available at: https://vlmsareblind.github.io/
State-of-the-art large language models with vision capabilities perform poorly on simple visual tasks, indicating limitations in their visual understanding.
large language models vision capabilities VLMs GPT-4o Gemini 1.5 Pro visual tasks visual understanding benchmarks myopia visual details blindness educated guesses