SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic
  Understanding, Localization, and Dense Features
We introduce SigLIP 2, a family of new multilingual vision-language encoders
that build on the success of the original SigLIP. In this second iteration, we
extend the original image-text training objective with several prior,
independently developed techniques into a unified recipe -- this includes
captioning-based pretraining, self-supervised losses (self-distillation, masked
prediction) and online data curation. With these changes, SigLIP 2 models
outperform their SigLIP counterparts at all model scales in core capabilities,
including zero-shot classification, image-text retrieval, and transfer
performance when extracting visual representations for Vision-Language Models
(VLMs). Furthermore, the new training recipe leads to significant improvements
on localization and dense prediction tasks. We also train variants which
support multiple resolutions and preserve the input's native aspect ratio.
Finally, we train on a more diverse data-mixture that includes de-biasing
techniques, leading to much better multilingual understanding and improved
fairness. To allow users to trade off inference cost with performance, we
release model checkpoints at four sizes: ViT-B (86M), L (303M), So400m (400M),
and g (1B).
67b7ed0d58f6b70b18dda7b4
Michael Tschannen
admin_assigned
2025-02-21T10:23:44.125Z
6489893e1ec8356ba5bb9777
/avatars/54354c1e5774cadd1d83d42054e9d96b.svg
Michael Tschannen
user
mitsch
67b7ed0d58f6b70b18dda7b5
Alexey Gritsenko
admin_assigned
2025-02-21T10:23:50.440Z
62d8f9887b8dc0ba17271415
/avatars/12ec78d34fd849bad44217b212f31e98.svg
Alexey Gritsenko
user
AlexeyG
67b7ed0d58f6b70b18dda7b6
Xiao Wang
67b7ed0d58f6b70b18dda7b7
Muhammad Ferjad Naeem
claimed_verified
2025-02-26T15:37:42.802Z
67bf33a512368ec2fad4fe29
/avatars/ea5c03744ec1c2bcc0e6c13efc8f7ddc.svg
Muhammad Ferjad Naeem
user
ferjad
67b7ed0d58f6b70b18dda7b8
Ibrahim Alabdulmohsin
admin_assigned
2025-02-21T10:24:03.702Z
630545da20668afe24860235
/avatars/5d82be2e7412bff1af15cc5eafa60b7d.svg
Ibrahim Alabdulmohsin
user
ibomohsin
67b7ed0d58f6b70b18dda7b9
Nikhil Parthasarathy
67b7ed0d58f6b70b18dda7ba
Talfan Evans
admin_assigned
2025-02-21T10:24:14.273Z
66be3074dac7d71326f613cf
/avatars/012a531aad3eb1a2751bb3c31a619bf5.svg
Talfan Evans
user
talfanevans
67b7ed0d58f6b70b18dda7bb
Lucas Beyer
admin_assigned
2025-02-21T14:44:51.778Z
642d334ff65714b4585f2de4
https://cdn-avatars.huggingface.co/v1/production/uploads/642d334ff65714b4585f2de4/gxBynq5KyoUP0VlAQD3-w.jpeg
Lucas Beyer
user
giffmana
67b7ed0d58f6b70b18dda7bc
Ye Xia
67b7ed0d58f6b70b18dda7bd
Basil Mustafa
admin_assigned
2025-02-21T14:44:44.933Z
63cfcad6e23b90128c66685c
https://cdn-avatars.huggingface.co/v1/production/uploads/1674562206311-noauth.png
Basil Mustafa
user
BasilMustafa
67b7ed0d58f6b70b18dda7be
Olivier Hénaff
admin_assigned
2025-02-21T14:44:38.156Z
64f30d8ceb5f2982081db604
/avatars/eedf65a104d099d8a60bbffe69bc2571.svg
Olivier Henaff
user
olivierhenaff
67b7ed0d58f6b70b18dda7bf
Jeremiah Harmsen
admin_assigned
2025-02-21T14:44:32.302Z
65d77c0e1e7686c460255fda
/avatars/1d26a7a7ffdc5ca9e67b97030f21b098.svg
Jeremiah Harmsen
user
jharmsen
67b7ed0d58f6b70b18dda7c0
Andreas Steiner
67b7ed0d58f6b70b18dda7c1
Xiaohua Zhai
admin_assigned
2025-02-21T14:44:07.595Z
65dcd90082bddd501f68174b
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/M2bc9PyKeFs1cCXjTfGGq.jpeg
Xiaohua Zhai
user
xiaohuazhai
SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic
  Understanding, Localization, and Dense Features
We introduce SigLIP 2, a family of new multilingual vision-language encoders
that build on the success of the original SigLIP. In this second iteration, we
extend the original image-text training objective with several prior,
independently developed techniques into a unified recipe -- this includes
captioning-based pretraining, self-supervised losses (self-distillation, masked
prediction) and online data curation. With these changes, SigLIP 2 models
outperform their SigLIP counterparts at all model scales in core capabilities,
including zero-shot classification, image-text retrieval, and transfer
performance when extracting visual representations for Vision-Language Models
(VLMs). Furthermore, the new training recipe leads to significant improvements
on localization and dense prediction tasks. We also train variants which
support multiple resolutions and preserve the input's native aspect ratio.
Finally, we train on a more diverse data-mixture that includes de-biasing
techniques, leading to much better multilingual understanding and improved
fairness. To allow users to trade off inference cost with performance, we
release model checkpoints at four sizes: ViT-B (86M), L (303M), So400m (400M),
and g (1B).
SigLIP 2, a multilingual vision-language encoder, improves upon SigLIP with unified training techniques, enhancing performance in zero-shot classification, image-text retrieval, localization, and dense prediction across various model sizes and data diversity.
vision-language encoders captioning-based pretraining self-supervised losses self-distillation masked prediction online data curation zero-shot classification image-text retrieval Visual-Language Models (VLMs) localization dense prediction multiple resolutions native aspect ratio de-biasing techniques multilingual understanding fairness model checkpoints ViT-B ViT-L So400m g