Adapting Vision-Language Models Without Labels: A Comprehensive Survey
Vision-Language Models (VLMs) have demonstrated remarkable generalization
capabilities across a wide range of tasks. However, their performance often
remains suboptimal when directly applied to specific downstream scenarios
without task-specific adaptation. To enhance their utility while preserving
data efficiency, recent research has increasingly focused on unsupervised
adaptation methods that do not rely on labeled data. Despite the growing
interest in this area, there remains a lack of a unified, task-oriented survey
dedicated to unsupervised VLM adaptation. To bridge this gap, we present a
comprehensive and structured overview of the field. We propose a taxonomy based
on the availability and nature of unlabeled visual data, categorizing existing
approaches into four key paradigms: Data-Free Transfer (no data), Unsupervised
Domain Transfer (abundant data), Episodic Test-Time Adaptation (batch data),
and Online Test-Time Adaptation (streaming data). Within this framework, we
analyze core methodologies and adaptation strategies associated with each
paradigm, aiming to establish a systematic understanding of the field.
Additionally, we review representative benchmarks across diverse applications
and highlight open challenges and promising directions for future research. An
actively maintained repository of relevant literature is available at
https://github.com/tim-learn/Awesome-LabelFree-VLMs.
689979a7f022d141f5d4360e
Hao Dong
claimed_verified
2025-08-13T07:26:30.259Z
6649fb62a460da1da20f66d0
/avatars/6afa26922ba8abe8c9603e6f7222531b.svg
Hao Dong
user
hdong51
689979a7f022d141f5d4360f
Lijun Sheng
689979a7f022d141f5d43610
Jian Liang
689979a7f022d141f5d43611
Ran He
689979a7f022d141f5d43612
Eleni Chatzi
689979a7f022d141f5d43613
Olga Fink
Adapting Vision-Language Models Without Labels: A Comprehensive Survey
Vision-Language Models (VLMs) have demonstrated remarkable generalization
capabilities across a wide range of tasks. However, their performance often
remains suboptimal when directly applied to specific downstream scenarios
without task-specific adaptation. To enhance their utility while preserving
data efficiency, recent research has increasingly focused on unsupervised
adaptation methods that do not rely on labeled data. Despite the growing
interest in this area, there remains a lack of a unified, task-oriented survey
dedicated to unsupervised VLM adaptation. To bridge this gap, we present a
comprehensive and structured overview of the field. We propose a taxonomy based
on the availability and nature of unlabeled visual data, categorizing existing
approaches into four key paradigms: Data-Free Transfer (no data), Unsupervised
Domain Transfer (abundant data), Episodic Test-Time Adaptation (batch data),
and Online Test-Time Adaptation (streaming data). Within this framework, we
analyze core methodologies and adaptation strategies associated with each
paradigm, aiming to establish a systematic understanding of the field.
Additionally, we review representative benchmarks across diverse applications
and highlight open challenges and promising directions for future research. An
actively maintained repository of relevant literature is available at
https://github.com/tim-learn/Awesome-LabelFree-VLMs.
A comprehensive survey of unsupervised adaptation methods for Vision-Language Models (VLMs) categorizes approaches based on the availability of unlabeled visual data and discusses methodologies, benchmarks, and future research directions.
Vision-Language Models unsupervised adaptation Data-Free Transfer Unsupervised Domain Transfer Episodic Test-Time Adaptation Online Test-Time Adaptation