MMPB: It's Time for Multi-Modal Personalization
Visual personalization is essential in user-facing AI systems such as smart
homes and healthcare, where aligning model behavior with user-centric concepts
is critical. However, recent large Vision-Language Models (VLMs), despite their
broad applicability, remain underexplored in their ability to adapt to
individual users. In this paper, we introduce MMPB, the first extensive
benchmark for evaluating VLMs on personalization. MMPB comprises 10k
image-query pairs and includes 111 personalizable concepts across four
categories: humans, animals, objects, and characters, with the human category
enriched with preference-grounded queries. We structure personalization into
three main task types, each highlighting a different key property of VLMs.
Using 23 widely used VLMs including both open- and closed-source models, we
evaluate personalization performance via a three-stage protocol: concept
injection, multi-turn dialogue, and personalized querying. Our findings
indicate that most VLMs (including some closed-source models) struggle with
personalization, particularly in maintaining consistency over dialogue,
handling user preferences, and adapting to visual cues. Our analysis reveals
that the challenges in VLM personalization (such as refusal behaviors and
long-context forgetting) highlight substantial room for improvement. By
identifying these limitations and offering a scalable benchmark, MMPB offers
valuable insights and a solid foundation for future research toward truly
personalized multi-modal AI. Project Page: aidaslab.github.io/MMPB
68db5230d2bf1f4b15ec74df
Jaeik Kim
claimed_verified
2025-09-30T06:00:49.584Z
67c6c1b3806c89e23ea4b42d
/avatars/29c88684f243b063d221ef29300ed18d.svg
Kim Jaeik
user
jaeikkim
68db5230d2bf1f4b15ec74e0
Woojin Kim
68db5230d2bf1f4b15ec74e1
Woohyeon Park
68db5230d2bf1f4b15ec74e2
Jaeyoung Do
MMPB: It's Time for Multi-Modal Personalization
Visual personalization is essential in user-facing AI systems such as smart
homes and healthcare, where aligning model behavior with user-centric concepts
is critical. However, recent large Vision-Language Models (VLMs), despite their
broad applicability, remain underexplored in their ability to adapt to
individual users. In this paper, we introduce MMPB, the first extensive
benchmark for evaluating VLMs on personalization. MMPB comprises 10k
image-query pairs and includes 111 personalizable concepts across four
categories: humans, animals, objects, and characters, with the human category
enriched with preference-grounded queries. We structure personalization into
three main task types, each highlighting a different key property of VLMs.
Using 23 widely used VLMs including both open- and closed-source models, we
evaluate personalization performance via a three-stage protocol: concept
injection, multi-turn dialogue, and personalized querying. Our findings
indicate that most VLMs (including some closed-source models) struggle with
personalization, particularly in maintaining consistency over dialogue,
handling user preferences, and adapting to visual cues. Our analysis reveals
that the challenges in VLM personalization (such as refusal behaviors and
long-context forgetting) highlight substantial room for improvement. By
identifying these limitations and offering a scalable benchmark, MMPB offers
valuable insights and a solid foundation for future research toward truly
personalized multi-modal AI. Project Page: aidaslab.github.io/MMPB
MMPB is a benchmark for evaluating the personalization capabilities of Vision-Language Models across various tasks and concepts, revealing significant challenges in maintaining consistency and adapting to user preferences.
Vision-Language Models VLMs personalization benchmark image-query pairs preference-grounded queries concept injection multi-turn dialogue personalized querying refusal behaviors long-context forgetting