SQUARE: Semantic Query-Augmented Fusion and Efficient Batch Reranking
  for Training-free Zero-Shot Composed Image Retrieval
Composed Image Retrieval (CIR) aims to retrieve target images that preserve
the visual content of a reference image while incorporating user-specified
textual modifications. Training-free zero-shot CIR (ZS-CIR) approaches, which
require no task-specific training or labeled data, are highly desirable, yet
accurately capturing user intent remains challenging. In this paper, we present
SQUARE, a novel two-stage training-free framework that leverages Multimodal
Large Language Models (MLLMs) to enhance ZS-CIR. In the Semantic
Query-Augmented Fusion (SQAF) stage, we enrich the query embedding derived from
a vision-language model (VLM) such as CLIP with MLLM-generated captions of the
target image. These captions provide high-level semantic guidance, enabling the
query to better capture the user's intent and improve global retrieval quality.
In the Efficient Batch Reranking (EBR) stage, top-ranked candidates are
presented as an image grid with visual marks to the MLLM, which performs joint
visual-semantic reasoning across all candidates. Our reranking strategy
operates in a single pass and yields more accurate rankings. Experiments show
that SQUARE, with its simplicity and effectiveness, delivers strong performance
on four standard CIR benchmarks. Notably, it maintains high performance even
with lightweight pre-trained, demonstrating its potential applicability.
68dc8d234159d1f2418f9a90
Ren-Di Wu
claimed_verified
2025-10-03T12:55:10.114Z
664ddfff6beee47c792f5836
https://cdn-avatars.huggingface.co/v1/production/uploads/664ddfff6beee47c792f5836/Wd0LIMCiHc9fq5iTslYgb.png
Ren-Di Wu
user
whats2000
68dc8d234159d1f2418f9a91
Yu-Yen Lin
68dc8d234159d1f2418f9a92
Huei-Fang Yang
SQUARE: Semantic Query-Augmented Fusion and Efficient Batch Reranking
  for Training-free Zero-Shot Composed Image Retrieval
Composed Image Retrieval (CIR) aims to retrieve target images that preserve
the visual content of a reference image while incorporating user-specified
textual modifications. Training-free zero-shot CIR (ZS-CIR) approaches, which
require no task-specific training or labeled data, are highly desirable, yet
accurately capturing user intent remains challenging. In this paper, we present
SQUARE, a novel two-stage training-free framework that leverages Multimodal
Large Language Models (MLLMs) to enhance ZS-CIR. In the Semantic
Query-Augmented Fusion (SQAF) stage, we enrich the query embedding derived from
a vision-language model (VLM) such as CLIP with MLLM-generated captions of the
target image. These captions provide high-level semantic guidance, enabling the
query to better capture the user's intent and improve global retrieval quality.
In the Efficient Batch Reranking (EBR) stage, top-ranked candidates are
presented as an image grid with visual marks to the MLLM, which performs joint
visual-semantic reasoning across all candidates. Our reranking strategy
operates in a single pass and yields more accurate rankings. Experiments show
that SQUARE, with its simplicity and effectiveness, delivers strong performance
on four standard CIR benchmarks. Notably, it maintains high performance even
with lightweight pre-trained, demonstrating its potential applicability.
SQUARE is a two-stage training-free framework that uses Multimodal Large Language Models to enhance zero-shot Composed Image Retrieval by enriching query embeddings and performing efficient batch reranking.
Multimodal Large Language Models MLLMs vision-language model VLM CLIP Semantic Query-Augmented Fusion SQAF Efficient Batch Reranking EBR joint visual-semantic reasoning Composed Image Retrieval CIR