PIA: Your Personalized Image Animator via Plug-and-Play Modules in
  Text-to-Image Models
Recent advancements in personalized text-to-image (T2I) models have
revolutionized content creation, empowering non-experts to generate stunning
images with unique styles. While promising, adding realistic motions into these
personalized images by text poses significant challenges in preserving distinct
styles, high-fidelity details, and achieving motion controllability by text. In
this paper, we present PIA, a Personalized Image Animator that excels in
aligning with condition images, achieving motion controllability by text, and
the compatibility with various personalized T2I models without specific tuning.
To achieve these goals, PIA builds upon a base T2I model with well-trained
temporal alignment layers, allowing for the seamless transformation of any
personalized T2I model into an image animation model. A key component of PIA is
the introduction of the condition module, which utilizes the condition frame
and inter-frame affinity as input to transfer appearance information guided by
the affinity hint for individual frame synthesis in the latent space. This
design mitigates the challenges of appearance-related image alignment within
and allows for a stronger focus on aligning with motion-related guidance.
6584f9100f26133956091593
Yiming Zhang
admin_assigned
2024-07-02T20:08:34.819Z
64ccaa4687ec96aa4752e754
/avatars/d2dd2040a521de4f55c7335cb7771c75.svg
Yiming Zhang
user
ymzhang319
6584f9100f26133956091594
Zhening Xing
admin_assigned
2023-12-22T12:30:04.341Z
62fb2a9dc95d426ff8f74c8d
/avatars/25c1a68ee7b7d0cc7e9f56bde37f4914.svg
Zhening Xing
user
Leoxing
6584f9100f26133956091595
Yanhong Zeng
claimed_verified
2023-12-22T13:00:39.833Z
63d4b843df01ef426a0f79fb
https://cdn-avatars.huggingface.co/v1/production/uploads/1676365795587-63d4b843df01ef426a0f79fb.jpeg
Yanhong Zeng
user
zengyh1900
6584f9100f26133956091596
Youqing Fang
6584f9100f26133956091597
Kai Chen
PIA: Your Personalized Image Animator via Plug-and-Play Modules in
  Text-to-Image Models
Recent advancements in personalized text-to-image (T2I) models have
revolutionized content creation, empowering non-experts to generate stunning
images with unique styles. While promising, adding realistic motions into these
personalized images by text poses significant challenges in preserving distinct
styles, high-fidelity details, and achieving motion controllability by text. In
this paper, we present PIA, a Personalized Image Animator that excels in
aligning with condition images, achieving motion controllability by text, and
the compatibility with various personalized T2I models without specific tuning.
To achieve these goals, PIA builds upon a base T2I model with well-trained
temporal alignment layers, allowing for the seamless transformation of any
personalized T2I model into an image animation model. A key component of PIA is
the introduction of the condition module, which utilizes the condition frame
and inter-frame affinity as input to transfer appearance information guided by
the affinity hint for individual frame synthesis in the latent space. This
design mitigates the challenges of appearance-related image alignment within
and allows for a stronger focus on aligning with motion-related guidance.
PIA enhances personalized T2I models to generate animations by integrating temporal alignment and condition modules for motion control and appearance consistency.
personalized T2I models temporal alignment layers condition module condition frame inter-frame affinity latent space image animation model motion controllability appearance-related image alignment