UniPose: A Unified Multimodal Framework for Human Pose Comprehension,
  Generation and Editing
Human pose plays a crucial role in the digital age. While recent works have
achieved impressive progress in understanding and generating human poses, they
often support only a single modality of control signals and operate in
isolation, limiting their application in real-world scenarios. This paper
presents UniPose, a framework employing Large Language Models (LLMs) to
comprehend, generate, and edit human poses across various modalities, including
images, text, and 3D SMPL poses. Specifically, we apply a pose tokenizer to
convert 3D poses into discrete pose tokens, enabling seamless integration into
the LLM within a unified vocabulary. To further enhance the fine-grained pose
perception capabilities, we facilitate UniPose with a mixture of visual
encoders, among them a pose-specific visual encoder. Benefiting from a unified
learning strategy, UniPose effectively transfers knowledge across different
pose-relevant tasks, adapts to unseen tasks, and exhibits extended
capabilities. This work serves as the first attempt at building a
general-purpose framework for pose comprehension, generation, and editing.
Extensive experiments highlight UniPose's competitive and even superior
performance across various pose-relevant tasks.
674713a34295e90fc523ab3d
Yiheng Li
claimed_verified
2024-11-27T22:19:41.408Z
66afcc9b3cbe4ea9a4b57df6
/avatars/add1c3cdfbb9c1013820761057a6a643.svg
Li YiHeng
user
L-yiheng
674713a34295e90fc523ab3e
Ruibing Hou
admin_assigned
2024-11-28T11:37:17.826Z
64f1e1badd92641727444364
/avatars/a2e9331b456694ff5f955281416e33d9.svg
ruibing hou
user
flow2023
674713a34295e90fc523ab3f
Hong Chang
674713a34295e90fc523ab40
Shiguang Shan
674713a34295e90fc523ab41
Xilin Chen
admin_assigned
2024-11-28T11:37:38.181Z
670f0d33c9504aac1179cd3c
/avatars/a593778043e9dca533cfb89e03ce57f9.svg
xilin chen
user
Heylon
UniPose: A Unified Multimodal Framework for Human Pose Comprehension,
  Generation and Editing
Human pose plays a crucial role in the digital age. While recent works have
achieved impressive progress in understanding and generating human poses, they
often support only a single modality of control signals and operate in
isolation, limiting their application in real-world scenarios. This paper
presents UniPose, a framework employing Large Language Models (LLMs) to
comprehend, generate, and edit human poses across various modalities, including
images, text, and 3D SMPL poses. Specifically, we apply a pose tokenizer to
convert 3D poses into discrete pose tokens, enabling seamless integration into
the LLM within a unified vocabulary. To further enhance the fine-grained pose
perception capabilities, we facilitate UniPose with a mixture of visual
encoders, among them a pose-specific visual encoder. Benefiting from a unified
learning strategy, UniPose effectively transfers knowledge across different
pose-relevant tasks, adapts to unseen tasks, and exhibits extended
capabilities. This work serves as the first attempt at building a
general-purpose framework for pose comprehension, generation, and editing.
Extensive experiments highlight UniPose's competitive and even superior
performance across various pose-relevant tasks.
UniPose, leveraging Large Language Models and unified visual encoders, comprehends, generates, and edits human poses across multiple modalities with superior performance.
Large Language Models LLMs pose tokenizer discrete pose tokens visual encoders pose-specific visual encoder unified learning strategy pose-relevant tasks