RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image
  Generation
Subject-driven text-to-image (T2I) generation aims to produce images that
align with a given textual description, while preserving the visual identity
from a referenced subject image. Despite its broad downstream applicability --
ranging from enhanced personalization in image generation to consistent
character representation in video rendering -- progress in this field is
limited by the lack of reliable automatic evaluation. Existing methods either
assess only one aspect of the task (i.e., textual alignment or subject
preservation), misalign with human judgments, or rely on costly API-based
evaluation. To address this, we introduce RefVNLI, a cost-effective metric that
evaluates both textual alignment and subject preservation in a single
prediction. Trained on a large-scale dataset derived from video-reasoning
benchmarks and image perturbations, RefVNLI outperforms or matches existing
baselines across multiple benchmarks and subject categories (e.g.,
Animal, Object), achieving up to 6.4-point gains in textual
alignment and 8.5-point gains in subject consistency. It also excels with
lesser-known concepts, aligning with human preferences at over 87\% accuracy.
680b44fb426b7d5bc2018c75
Aviv Slobodkin
extracted_confirmed
2025-06-07T08:09:16.083Z
631da07f6d6a5870f3d2c375
/avatars/242e344dca08057bdf1eef09f69b41b2.svg
Aviv Slobodkin
user
lovodkin93
680b44fb426b7d5bc2018c76
Hagai Taitelbaum
680b44fb426b7d5bc2018c77
Yonatan Bitton
680b44fb426b7d5bc2018c78
Brian Gordon
680b44fb426b7d5bc2018c79
Michal Sokolik
680b44fb426b7d5bc2018c7a
Nitzan Bitton Guetta
680b44fb426b7d5bc2018c7b
Almog Gueta
680b44fb426b7d5bc2018c7c
Royi Rassin
680b44fb426b7d5bc2018c7d
Itay Laish
680b44fb426b7d5bc2018c7e
Dani Lischinski
680b44fb426b7d5bc2018c7f
Idan Szpektor
RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image
  Generation
Subject-driven text-to-image (T2I) generation aims to produce images that
align with a given textual description, while preserving the visual identity
from a referenced subject image. Despite its broad downstream applicability --
ranging from enhanced personalization in image generation to consistent
character representation in video rendering -- progress in this field is
limited by the lack of reliable automatic evaluation. Existing methods either
assess only one aspect of the task (i.e., textual alignment or subject
preservation), misalign with human judgments, or rely on costly API-based
evaluation. To address this, we introduce RefVNLI, a cost-effective metric that
evaluates both textual alignment and subject preservation in a single
prediction. Trained on a large-scale dataset derived from video-reasoning
benchmarks and image perturbations, RefVNLI outperforms or matches existing
baselines across multiple benchmarks and subject categories (e.g.,
Animal, Object), achieving up to 6.4-point gains in textual
alignment and 8.5-point gains in subject consistency. It also excels with
lesser-known concepts, aligning with human preferences at over 87\% accuracy.
RefVNLI, a novel metric, evaluates both textual alignment and subject preservation in subject-driven text-to-image generation, outperforming existing methods across various benchmarks and subject categories.
text-to-image T2I textual alignment subject preservation RefVNLI video-reasoning benchmarks image perturbations