To Trust Or Not To Trust Your Vision-Language Model's Prediction
Vision-Language Models (VLMs) have demonstrated strong capabilities in
aligning visual and textual modalities, enabling a wide range of applications
in multimodal understanding and generation. While they excel in zero-shot and
transfer learning scenarios, VLMs remain susceptible to misclassification,
often yielding confident yet incorrect predictions. This limitation poses a
significant risk in safety-critical domains, where erroneous predictions can
lead to severe consequences. In this work, we introduce TrustVLM, a
training-free framework designed to address the critical challenge of
estimating when VLM's predictions can be trusted. Motivated by the observed
modality gap in VLMs and the insight that certain concepts are more distinctly
represented in the image embedding space, we propose a novel confidence-scoring
function that leverages this space to improve misclassification detection. We
rigorously evaluate our approach across 17 diverse datasets, employing 4
architectures and 2 VLMs, and demonstrate state-of-the-art performance, with
improvements of up to 51.87% in AURC, 9.14% in AUROC, and 32.42% in FPR95
compared to existing baselines. By improving the reliability of the model
without requiring retraining, TrustVLM paves the way for safer deployment of
VLMs in real-world applications. The code will be available at
https://github.com/EPFL-IMOS/TrustVLM.
683910d50df60182c0dd5b62
Hao Dong
claimed_verified
2025-05-30T06:53:56.719Z
6649fb62a460da1da20f66d0
/avatars/6afa26922ba8abe8c9603e6f7222531b.svg
Hao Dong
user
hdong51
683910d50df60182c0dd5b63
Moru Liu
683910d50df60182c0dd5b64
Jian Liang
683910d50df60182c0dd5b65
Eleni Chatzi
683910d50df60182c0dd5b66
Olga Fink
To Trust Or Not To Trust Your Vision-Language Model's Prediction
Vision-Language Models (VLMs) have demonstrated strong capabilities in
aligning visual and textual modalities, enabling a wide range of applications
in multimodal understanding and generation. While they excel in zero-shot and
transfer learning scenarios, VLMs remain susceptible to misclassification,
often yielding confident yet incorrect predictions. This limitation poses a
significant risk in safety-critical domains, where erroneous predictions can
lead to severe consequences. In this work, we introduce TrustVLM, a
training-free framework designed to address the critical challenge of
estimating when VLM's predictions can be trusted. Motivated by the observed
modality gap in VLMs and the insight that certain concepts are more distinctly
represented in the image embedding space, we propose a novel confidence-scoring
function that leverages this space to improve misclassification detection. We
rigorously evaluate our approach across 17 diverse datasets, employing 4
architectures and 2 VLMs, and demonstrate state-of-the-art performance, with
improvements of up to 51.87% in AURC, 9.14% in AUROC, and 32.42% in FPR95
compared to existing baselines. By improving the reliability of the model
without requiring retraining, TrustVLM paves the way for safer deployment of
VLMs in real-world applications. The code will be available at
https://github.com/EPFL-IMOS/TrustVLM.
TrustVLM enhances the reliability of Vision-Language Models by estimating prediction trustworthiness without retraining, improving misclassification detection in multimodal tasks.
Vision-Language Models VLMs modality gap image embedding space confidence-scoring function AURC AUROC FPR95