ILIAS: Instance-Level Image retrieval At Scale
This work introduces ILIAS, a new test dataset for Instance-Level Image
retrieval At Scale. It is designed to evaluate the ability of current and
future foundation models and retrieval techniques to recognize particular
objects. The key benefits over existing datasets include large scale, domain
diversity, accurate ground truth, and a performance that is far from saturated.
ILIAS includes query and positive images for 1,000 object instances, manually
collected to capture challenging conditions and diverse domains. Large-scale
retrieval is conducted against 100 million distractor images from YFCC100M. To
avoid false negatives without extra annotation effort, we include only query
objects confirmed to have emerged after 2014, i.e. the compilation date of
YFCC100M. An extensive benchmarking is performed with the following
observations: i) models fine-tuned on specific domains, such as landmarks or
products, excel in that domain but fail on ILIAS ii) learning a linear
adaptation layer using multi-domain class supervision results in performance
improvements, especially for vision-language models iii) local descriptors in
retrieval re-ranking are still a key ingredient, especially in the presence of
severe background clutter iv) the text-to-image performance of the
vision-language foundation models is surprisingly close to the corresponding
image-to-image case. website: https://vrg.fel.cvut.cz/ilias/
67b465600e5142133055d7c1
Giorgos Kordopatis-Zilos
claimed_verified
2025-02-19T09:04:38.791Z
673c934bdf13003bd11746fd
/avatars/1aec1157549be85963b39eb54845b695.svg
Giorgos Kordopatis-Zilos
user
gkordo
67b465600e5142133055d7c2
Vladan Stojnić
claimed_verified
2025-02-18T16:39:41.653Z
66a3ae59f33ff23e1c027ccd
https://cdn-avatars.huggingface.co/v1/production/uploads/66a3ae59f33ff23e1c027ccd/tZzpESNPnmhty62xhHszF.jpeg
Vladan Stojnic
user
stojnvla
67b465600e5142133055d7c3
Anna Manko
67b465600e5142133055d7c4
Pavel Šuma
claimed_verified
2025-02-19T09:04:41.052Z
67ab9acd412358244419d946
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/tq-eUQMYGdlDZVn15DckY.png
Pavel Suma
user
pavelsuma
67b465600e5142133055d7c5
Nikolaos-Antonios Ypsilantis
67b465600e5142133055d7c6
Nikos Efthymiadis
claimed_verified
2025-02-19T11:49:36.494Z
663e63c5902b965ba35a0308
https://cdn-avatars.huggingface.co/v1/production/uploads/663e63c5902b965ba35a0308/OFX5yvY2LL8KORKsiDhAy.jpeg
Nikos Efthymiadis
user
nikos-efth
67b465600e5142133055d7c7
Zakaria Laskar
67b465600e5142133055d7c8
Jiří Matas
67b465600e5142133055d7c9
Ondřej Chum
67b465600e5142133055d7ca
Giorgos Tolias
ILIAS: Instance-Level Image retrieval At Scale
This work introduces ILIAS, a new test dataset for Instance-Level Image
retrieval At Scale. It is designed to evaluate the ability of current and
future foundation models and retrieval techniques to recognize particular
objects. The key benefits over existing datasets include large scale, domain
diversity, accurate ground truth, and a performance that is far from saturated.
ILIAS includes query and positive images for 1,000 object instances, manually
collected to capture challenging conditions and diverse domains. Large-scale
retrieval is conducted against 100 million distractor images from YFCC100M. To
avoid false negatives without extra annotation effort, we include only query
objects confirmed to have emerged after 2014, i.e. the compilation date of
YFCC100M. An extensive benchmarking is performed with the following
observations: i) models fine-tuned on specific domains, such as landmarks or
products, excel in that domain but fail on ILIAS ii) learning a linear
adaptation layer using multi-domain class supervision results in performance
improvements, especially for vision-language models iii) local descriptors in
retrieval re-ranking are still a key ingredient, especially in the presence of
severe background clutter iv) the text-to-image performance of the
vision-language foundation models is surprisingly close to the corresponding
image-to-image case. website: https://vrg.fel.cvut.cz/ilias/
ILIAS, a new large-scale and diverse dataset, evaluates instance-level image retrieval capabilities and highlights the performance of domain-specific and vision-language models.
Instance-Level Image retrieval ILIAS foundation models retrieval techniques ground truth YFCC100M multi-domain class supervision local descriptors vision-language models text-to-image image-to-image