LaMP-Cap: Personalized Figure Caption Generation With Multimodal Figure
  Profiles
Figure captions are crucial for helping readers understand and remember a
figure's key message. Many models have been developed to generate these
captions, helping authors compose better quality captions more easily. Yet,
authors almost always need to revise generic AI-generated captions to match
their writing style and the domain's style, highlighting the need for
personalization. Despite language models' personalization (LaMP) advances,
these technologies often focus on text-only settings and rarely address
scenarios where both inputs and profiles are multimodal. This paper introduces
LaMP-Cap, a dataset for personalized figure caption generation with multimodal
figure profiles. For each target figure, LaMP-Cap provides not only the needed
inputs, such as figure images, but also up to three other figures from the same
document--each with its image, caption, and figure-mentioning paragraphs--as a
profile to characterize the context. Experiments with four LLMs show that using
profile information consistently helps generate captions closer to the original
author-written ones. Ablation studies reveal that images in the profile are
more helpful than figure-mentioning paragraphs, highlighting the advantage of
using multimodal profiles over text-only ones.
684b88113b733ba333686fc7
Ho Yin 'Sam' Ng
684b88113b733ba333686fc8
Ting-Yao Hsu
684b88113b733ba333686fc9
Aashish Anantha Ramakrishnan
684b88113b733ba333686fca
Branislav Kveton
684b88113b733ba333686fcb
Nedim Lipka
684b88113b733ba333686fcc
Franck Dernoncourt
claimed_verified
2025-06-13T07:39:49.692Z
62c5947524171688a9feb992
/avatars/5a151713b9eae8dc566f5957acee3475.svg
Franck Dernoncourt
user
Franck-Dernoncourt
684b88113b733ba333686fcd
Dongwon Lee
684b88113b733ba333686fce
Tong Yu
684b88113b733ba333686fcf
Sungchul Kim
684b88113b733ba333686fd0
Ryan A. Rossi
684b88113b733ba333686fd1
Ting-Hao 'Kenneth' Huang
LaMP-Cap: Personalized Figure Caption Generation With Multimodal Figure
  Profiles
Figure captions are crucial for helping readers understand and remember a
figure's key message. Many models have been developed to generate these
captions, helping authors compose better quality captions more easily. Yet,
authors almost always need to revise generic AI-generated captions to match
their writing style and the domain's style, highlighting the need for
personalization. Despite language models' personalization (LaMP) advances,
these technologies often focus on text-only settings and rarely address
scenarios where both inputs and profiles are multimodal. This paper introduces
LaMP-Cap, a dataset for personalized figure caption generation with multimodal
figure profiles. For each target figure, LaMP-Cap provides not only the needed
inputs, such as figure images, but also up to three other figures from the same
document--each with its image, caption, and figure-mentioning paragraphs--as a
profile to characterize the context. Experiments with four LLMs show that using
profile information consistently helps generate captions closer to the original
author-written ones. Ablation studies reveal that images in the profile are
more helpful than figure-mentioning paragraphs, highlighting the advantage of
using multimodal profiles over text-only ones.
LaMP-Cap introduces a dataset for personalized figure caption generation using multimodal profiles to improve the quality of AI-generated captions.
LaMP-Cap personalized figure caption generation multimodal figures figure profiles ablation studies