UniPose: A Unified Multimodal Framework for Human Pose Comprehension,
  Generation and Editing
Human pose plays a crucial role in the digital age. While recent works have
achieved impressive progress in understanding and generating human poses, they
often support only a single modality of control signals and operate in
isolation, limiting their application in real-world scenarios. This paper
presents UniPose, a framework employing Large Language Models (LLMs) to
comprehend, generate, and edit human poses across various modalities, including
images, text, and 3D SMPL poses. Specifically, we apply a pose tokenizer to
convert 3D poses into discrete pose tokens, enabling seamless integration into
the LLM within a unified vocabulary. To further enhance the fine-grained pose
perception capabilities, we facilitate UniPose with a mixture of visual
encoders, among them a pose-specific visual encoder. Benefiting from a unified
learning strategy, UniPose effectively transfers knowledge across different
pose-relevant tasks, adapts to unseen tasks, and exhibits extended
capabilities. This work serves as the first attempt at building a
general-purpose framework for pose comprehension, generation, and editing.
Extensive experiments highlight UniPose's competitive and even superior
performance across various pose-relevant tasks.
674713a34295e90fc523ab3d
Yiheng Li
claimed_verified
2024-11-27T22:19:41.408Z
66afcc9b3cbe4ea9a4b57df6
/avatars/add1c3cdfbb9c1013820761057a6a643.svg
Li YiHeng
user
L-yiheng
674713a34295e90fc523ab3e
Ruibing Hou
admin_assigned
2024-11-28T11:37:17.826Z
64f1e1badd92641727444364
/avatars/a2e9331b456694ff5f955281416e33d9.svg
ruibing hou
user
flow2023
674713a34295e90fc523ab3f
Hong Chang
674713a34295e90fc523ab40
Shiguang Shan
674713a34295e90fc523ab41
Xilin Chen
admin_assigned
2024-11-28T11:37:38.181Z
670f0d33c9504aac1179cd3c
/avatars/a593778043e9dca533cfb89e03ce57f9.svg
xilin chen
user
Heylon
UniPose: A Unified Multimodal Framework for Human Pose Comprehension,
  Generation and Editing
Human pose plays a crucial role in the digital age. While recent works have
achieved impressive progress in understanding and generating human poses, they
often support only a single modality of control signals and operate in
isolation, limiting their application in real-world scenarios. This paper
presents UniPose, a framework employing Large Language Models (LLMs) to
comprehend, generate, and edit human poses across various modalities, including
images, text, and 3D SMPL poses. Specifically, we apply a pose tokenizer to
convert 3D poses into discrete pose tokens, enabling seamless integration into
the LLM within a unified vocabulary. To further enhance the fine-grained pose
perception capabilities, we facilitate UniPose with a mixture of visual
encoders, among them a pose-specific visual encoder. Benefiting from a unified
learning strategy, UniPose effectively transfers knowledge across different
pose-relevant tasks, adapts to unseen tasks, and exhibits extended
capabilities. This work serves as the first attempt at building a
general-purpose framework for pose comprehension, generation, and editing.
Extensive experiments highlight UniPose's competitive and even superior
performance across various pose-relevant tasks.
Large Language Models (LLMs) pose tokenizer 3D SMPL poses discrete pose tokens unified vocabulary pose-specific visual encoder unified learning strategy