Video2Roleplay: A Multimodal Dataset and Framework for Video-Guided
  Role-playing Agents
Role-playing agents (RPAs) have attracted growing interest for their ability
to simulate immersive and interactive characters. However, existing approaches
primarily focus on static role profiles, overlooking the dynamic perceptual
abilities inherent to humans. To bridge this gap, we introduce the concept of
dynamic role profiles by incorporating video modality into RPAs. To support
this, we construct Role-playing-Video60k, a large-scale, high-quality dataset
comprising 60k videos and 700k corresponding dialogues. Based on this dataset,
we develop a comprehensive RPA framework that combines adaptive temporal
sampling with both dynamic and static role profile representations.
Specifically, the dynamic profile is created by adaptively sampling video
frames and feeding them to the LLM in temporal order, while the static profile
consists of (1) character dialogues from training videos during fine-tuning,
and (2) a summary context from the input video during inference. This joint
integration enables RPAs to generate greater responses. Furthermore, we propose
a robust evaluation method covering eight metrics. Experimental results
demonstrate the effectiveness of our framework, highlighting the importance of
dynamic role profiles in developing RPAs.
68d0af068adc5cd018d15ab4
Xueqiao Zhang
68d0af068adc5cd018d15ab5
Chao Zhang
68d0af068adc5cd018d15ab6
Jingtao Xu
admin_assigned
2025-09-22T03:04:25.303Z
666a674967c686801acf25bb
/avatars/c1f3edd63fd378dfb555e6413a966932.svg
jingtao xu
user
raul678
68d0af068adc5cd018d15ab7
Yifan Zhu
admin_assigned
2025-09-22T03:04:32.022Z
64e941a114c57101f39494a7
/avatars/f77aeb9ab8343e125b85db45431701ad.svg
Yifan Zhu
user
YifanZhu
68d0af068adc5cd018d15ab8
Xin Shi
68d0af068adc5cd018d15ab9
Yi Yang
68d0af068adc5cd018d15aba
Yawei Luo
admin_assigned
2025-09-22T03:04:11.110Z
665bfc4c88912c5ab646c3c9
/avatars/db284af3be943b5786cf0026279ef3b9.svg
Yawei Luo
user
RoyalVane
Video2Roleplay: A Multimodal Dataset and Framework for Video-Guided
  Role-playing Agents
Role-playing agents (RPAs) have attracted growing interest for their ability
to simulate immersive and interactive characters. However, existing approaches
primarily focus on static role profiles, overlooking the dynamic perceptual
abilities inherent to humans. To bridge this gap, we introduce the concept of
dynamic role profiles by incorporating video modality into RPAs. To support
this, we construct Role-playing-Video60k, a large-scale, high-quality dataset
comprising 60k videos and 700k corresponding dialogues. Based on this dataset,
we develop a comprehensive RPA framework that combines adaptive temporal
sampling with both dynamic and static role profile representations.
Specifically, the dynamic profile is created by adaptively sampling video
frames and feeding them to the LLM in temporal order, while the static profile
consists of (1) character dialogues from training videos during fine-tuning,
and (2) a summary context from the input video during inference. This joint
integration enables RPAs to generate greater responses. Furthermore, we propose
a robust evaluation method covering eight metrics. Experimental results
demonstrate the effectiveness of our framework, highlighting the importance of
dynamic role profiles in developing RPAs.
A framework incorporating dynamic role profiles with video modality enhances role-playing agents by combining adaptive temporal sampling and static role profile representations, improving response generation.
role-playing agents dynamic role profiles video modality Role-playing-Video60k adaptive temporal sampling LLM static role profile representations evaluation method