Semantic Library Adaptation: LoRA Retrieval and Fusion for
  Open-Vocabulary Semantic Segmentation
Open-vocabulary semantic segmentation models associate vision and text to
label pixels from an undefined set of classes using textual queries, providing
versatile performance on novel datasets. However, large shifts between training
and test domains degrade their performance, requiring fine-tuning for effective
real-world applications. We introduce Semantic Library Adaptation (SemLA), a
novel framework for training-free, test-time domain adaptation. SemLA leverages
a library of LoRA-based adapters indexed with CLIP embeddings, dynamically
merging the most relevant adapters based on proximity to the target domain in
the embedding space. This approach constructs an ad-hoc model tailored to each
specific input without additional training. Our method scales efficiently,
enhances explainability by tracking adapter contributions, and inherently
protects data privacy, making it ideal for sensitive applications.
Comprehensive experiments on a 20-domain benchmark built over 10 standard
datasets demonstrate SemLA's superior adaptability and performance across
diverse settings, establishing a new standard in domain adaptation for
open-vocabulary semantic segmentation.
67e6bcb5b07543ff130223c9
Reza Qorbani
claimed_verified
2025-03-31T08:14:45.530Z
643a48ec0e5495afdef66a79
/avatars/ffd0bd84a9741eb78638b44bfbd79b76.svg
Reza Qorbani
user
rezaqorbani
67e6bcb5b07543ff130223ca
Gianluca Villani
admin_assigned
2025-03-31T08:21:56.410Z
65e893ad531c1e29e47ae232
/avatars/8e818d7f4d764909249286d15cdee910.svg
Gianluca Villani
user
vprometheus
67e6bcb5b07543ff130223cb
Theodoros Panagiotakopoulos
admin_assigned
2025-03-31T08:22:02.228Z
64c3e47ee77ea9f2811523f4
/avatars/204fbd8e72f6208da56934cc64cee3a4.svg
Theodoros Panagiotakopoulos
user
theo2021
67e6bcb5b07543ff130223cc
Marc Botet Colomer
admin_assigned
2025-03-31T08:22:07.950Z
64c3d659ed521f27a4e4a3cd
/avatars/7ef00e391d61bb25234c7f304501b844.svg
Marc Botet Colomer
user
MarcBotet
67e6bcb5b07543ff130223cd
Linus Härenstam-Nielsen
admin_assigned
2025-03-31T08:22:18.276Z
645cf26ace72244df7b11787
/avatars/98daec0b19a45ebe5ac7798dd48ab7c9.svg
Linus Härenstam-Nielsen
user
Linusnie
67e6bcb5b07543ff130223ce
Mattia Segu
admin_assigned
2025-03-31T08:22:24.288Z
67178f9475b652a8eab5d23b
/avatars/da260d4d0f013404cbdb6aaf6929c871.svg
Mattia Segu
user
mattiasegu
67e6bcb5b07543ff130223cf
Pier Luigi Dovesi
admin_assigned
2025-03-31T08:22:31.498Z
6272b397d9ce7af3f4f95938
/avatars/1163a7d4ffcfa1cf4ffb6e01476b85df.svg
Pier Luigi Dovesi
user
dovpie
67e6bcb5b07543ff130223d0
Jussi Karlgren
admin_assigned
2025-03-31T08:22:37.417Z
6476103b95a4dc98e58bfc6c
https://cdn-avatars.huggingface.co/v1/production/uploads/6476103b95a4dc98e58bfc6c/45v7zaAjAYVyrVcfkJWSx.jpeg
Jussi  Karlgren
user
jussikarlgren
67e6bcb5b07543ff130223d1
Daniel Cremers
67e6bcb5b07543ff130223d2
Federico Tombari
67e6bcb5b07543ff130223d3
Matteo Poggi
admin_assigned
2025-03-31T08:22:50.955Z
6613b33d1dc41eabb76b8d82
/avatars/5c31d071c53dee4c19fcceb5b437c130.svg
Matteo Poggi
user
mpoggi
Semantic Library Adaptation: LoRA Retrieval and Fusion for
  Open-Vocabulary Semantic Segmentation
Open-vocabulary semantic segmentation models associate vision and text to
label pixels from an undefined set of classes using textual queries, providing
versatile performance on novel datasets. However, large shifts between training
and test domains degrade their performance, requiring fine-tuning for effective
real-world applications. We introduce Semantic Library Adaptation (SemLA), a
novel framework for training-free, test-time domain adaptation. SemLA leverages
a library of LoRA-based adapters indexed with CLIP embeddings, dynamically
merging the most relevant adapters based on proximity to the target domain in
the embedding space. This approach constructs an ad-hoc model tailored to each
specific input without additional training. Our method scales efficiently,
enhances explainability by tracking adapter contributions, and inherently
protects data privacy, making it ideal for sensitive applications.
Comprehensive experiments on a 20-domain benchmark built over 10 standard
datasets demonstrate SemLA's superior adaptability and performance across
diverse settings, establishing a new standard in domain adaptation for
open-vocabulary semantic segmentation.
Semantic Library Adaptation (SemLA) enhances open-vocabulary semantic segmentation with a train-free approach using LoRA adapters indexed by CLIP embeddings for real-world applications across various domains.
open-vocabulary semantic segmentation LoRA-based adapters CLIP embeddings domain adaptation semantic segmentation SemLA train-free adaptation model adaptation test-time adaptation adapter contributions data privacy