Bidirectional Likelihood Estimation with Multi-Modal Large Language
  Models for Text-Video Retrieval
Text-Video Retrieval aims to find the most relevant text (or video) candidate
given a video (or text) query from large-scale online databases. Recent work
leverages multi-modal large language models (MLLMs) to improve retrieval,
especially for long or complex query-candidate pairs. However, we observe that
the naive application of MLLMs, i.e., retrieval based on candidate likelihood,
introduces candidate prior bias, favoring candidates with inherently higher
priors over those more relevant to the query. To this end, we propose a novel
retrieval framework, Bidirectional Likelihood Estimation with MLLM (BLiM),
which leverages both query and candidate likelihoods by training the model to
generate text from a given video as well as video features from a given text.
Furthermore, we introduce Candidate Prior Normalization (CPN), a simple yet
effective training-free score calibration module designed to mitigate candidate
prior bias in candidate likelihood. On four Text-Video Retrieval benchmarks,
our BLiM equipped with CPN outperforms previous state-of-the-art models by 6.4
R@1 on average, effectively alleviating candidate prior bias and emphasizing
query-candidate relevance. Our in-depth analysis across various multi-modal
tasks beyond retrieval highlights the broad applicability of CPN which enhances
visual understanding by reducing reliance on textual priors. Code is available
at https://github.com/mlvlab/BLiM.
688c35758c434640078cc3db
Dohwan Ko
claimed_verified
2025-08-07T10:42:52.423Z
62bab5a69f045fd1fb9f760e
/avatars/3f12a908faec1135e0221444f0769496.svg
Dohwan Ko
user
ikodoh
688c35758c434640078cc3dc
Ji Soo Lee
688c35758c434640078cc3dd
Minhyuk Choi
688c35758c434640078cc3de
Zihang Meng
688c35758c434640078cc3df
Hyunwoo J. Kim
Bidirectional Likelihood Estimation with Multi-Modal Large Language
  Models for Text-Video Retrieval
Text-Video Retrieval aims to find the most relevant text (or video) candidate
given a video (or text) query from large-scale online databases. Recent work
leverages multi-modal large language models (MLLMs) to improve retrieval,
especially for long or complex query-candidate pairs. However, we observe that
the naive application of MLLMs, i.e., retrieval based on candidate likelihood,
introduces candidate prior bias, favoring candidates with inherently higher
priors over those more relevant to the query. To this end, we propose a novel
retrieval framework, Bidirectional Likelihood Estimation with MLLM (BLiM),
which leverages both query and candidate likelihoods by training the model to
generate text from a given video as well as video features from a given text.
Furthermore, we introduce Candidate Prior Normalization (CPN), a simple yet
effective training-free score calibration module designed to mitigate candidate
prior bias in candidate likelihood. On four Text-Video Retrieval benchmarks,
our BLiM equipped with CPN outperforms previous state-of-the-art models by 6.4
R@1 on average, effectively alleviating candidate prior bias and emphasizing
query-candidate relevance. Our in-depth analysis across various multi-modal
tasks beyond retrieval highlights the broad applicability of CPN which enhances
visual understanding by reducing reliance on textual priors. Code is available
at https://github.com/mlvlab/BLiM.
A novel retrieval framework using bidirectional likelihood estimation with multi-modal large language models and candidate prior normalization improves text-video retrieval by reducing candidate prior bias and enhancing query-candidate relevance.
multi-modal large language models MLLMs bidirectional likelihood estimation BLiM candidate prior bias candidate likelihood Candidate Prior Normalization CPN text-video retrieval R@1 visual understanding