PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits
Understanding human behavior traits is central to applications in
human-computer interaction, computational social science, and personalized AI
systems. Such understanding often requires integrating multiple modalities to
capture nuanced patterns and relationships. However, existing resources rarely
provide datasets that combine behavioral descriptors with complementary
modalities such as facial attributes and biographical information. To address
this gap, we present PersonaX, a curated collection of multimodal datasets
designed to enable comprehensive analysis of public traits across modalities.
PersonaX consists of (1) CelebPersona, featuring 9444 public figures from
diverse occupations, and (2) AthlePersona, covering 4181 professional athletes
across 7 major sports leagues. Each dataset includes behavioral trait
assessments inferred by three high-performing large language models, alongside
facial imagery and structured biographical features. We analyze PersonaX at two
complementary levels. First, we abstract high-level trait scores from text
descriptions and apply five statistical independence tests to examine their
relationships with other modalities. Second, we introduce a novel causal
representation learning (CRL) framework tailored to multimodal and
multi-measurement data, providing theoretical identifiability guarantees.
Experiments on both synthetic and real-world data demonstrate the effectiveness
of our approach. By unifying structured and unstructured analysis, PersonaX
establishes a foundation for studying LLM-inferred behavioral traits in
conjunction with visual and biographical attributes, advancing multimodal trait
analysis and causal reasoning.
68c8d175733e345e52ac1eb1
Loka Li
68c8d175733e345e52ac1eb2
Wong Yu Kang
68c8d175733e345e52ac1eb3
Minghao Fu
68c8d175733e345e52ac1eb4
Guangyi Chen
68c8d175733e345e52ac1eb5
Zhenhao Chen
68c8d175733e345e52ac1eb6
Gongxu Luo
68c8d175733e345e52ac1eb7
Yuewen Sun
68c8d175733e345e52ac1eb8
Salman Khan
68c8d175733e345e52ac1eb9
Peter Spirtes
68c8d175733e345e52ac1eba
Kun Zhang
PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits
Understanding human behavior traits is central to applications in
human-computer interaction, computational social science, and personalized AI
systems. Such understanding often requires integrating multiple modalities to
capture nuanced patterns and relationships. However, existing resources rarely
provide datasets that combine behavioral descriptors with complementary
modalities such as facial attributes and biographical information. To address
this gap, we present PersonaX, a curated collection of multimodal datasets
designed to enable comprehensive analysis of public traits across modalities.
PersonaX consists of (1) CelebPersona, featuring 9444 public figures from
diverse occupations, and (2) AthlePersona, covering 4181 professional athletes
across 7 major sports leagues. Each dataset includes behavioral trait
assessments inferred by three high-performing large language models, alongside
facial imagery and structured biographical features. We analyze PersonaX at two
complementary levels. First, we abstract high-level trait scores from text
descriptions and apply five statistical independence tests to examine their
relationships with other modalities. Second, we introduce a novel causal
representation learning (CRL) framework tailored to multimodal and
multi-measurement data, providing theoretical identifiability guarantees.
Experiments on both synthetic and real-world data demonstrate the effectiveness
of our approach. By unifying structured and unstructured analysis, PersonaX
establishes a foundation for studying LLM-inferred behavioral traits in
conjunction with visual and biographical attributes, advancing multimodal trait
analysis and causal reasoning.
PersonaX, a multimodal dataset, combines behavioral traits, facial imagery, and biographical information to enable comprehensive analysis and causal reasoning using large language models.
large language models causal representation learning multimodal datasets behavioral traits facial imagery biographical information statistical independence tests theoretical identifiability guarantees