Durian: Dual Reference-guided Portrait Animation with Attribute Transfer
We present Durian, the first method for generating portrait animation videos
with facial attribute transfer from a given reference image to a target
portrait in a zero-shot manner. To enable high-fidelity and spatially
consistent attribute transfer across frames, we introduce dual reference
networks that inject spatial features from both the portrait and attribute
images into the denoising process of a diffusion model. We train the model
using a self-reconstruction formulation, where two frames are sampled from the
same portrait video: one is treated as the attribute reference and the other as
the target portrait, and the remaining frames are reconstructed conditioned on
these inputs and their corresponding masks. To support the transfer of
attributes with varying spatial extent, we propose a mask expansion strategy
using keypoint-conditioned image generation for training. In addition, we
further augment the attribute and portrait images with spatial and
appearance-level transformations to improve robustness to positional
misalignment between them. These strategies allow the model to effectively
generalize across diverse attributes and in-the-wild reference combinations,
despite being trained without explicit triplet supervision. Durian achieves
state-of-the-art performance on portrait animation with attribute transfer, and
notably, its dual reference design enables multi-attribute composition in a
single generation pass without additional training.
68ba5959736018af705e8e93
Hyunsoo Cha
admin_assigned
2025-09-05T14:34:48.828Z
62414e2b585605a4079c2f38
/avatars/db1dc5dd2164b7ecbd789104329296bd.svg
Hyunsoo Cha
user
HyunsooCha
68ba5959736018af705e8e94
Byungjun Kim
68ba5959736018af705e8e95
Hanbyul Joo
admin_assigned
2025-09-05T14:34:59.097Z
65b0ebe1165e69e0e6dc6ddd
/avatars/d2e95b2040c761200df1293db2949da7.svg
Hanbyul Joo
user
jhugestar
Durian: Dual Reference-guided Portrait Animation with Attribute Transfer
We present Durian, the first method for generating portrait animation videos
with facial attribute transfer from a given reference image to a target
portrait in a zero-shot manner. To enable high-fidelity and spatially
consistent attribute transfer across frames, we introduce dual reference
networks that inject spatial features from both the portrait and attribute
images into the denoising process of a diffusion model. We train the model
using a self-reconstruction formulation, where two frames are sampled from the
same portrait video: one is treated as the attribute reference and the other as
the target portrait, and the remaining frames are reconstructed conditioned on
these inputs and their corresponding masks. To support the transfer of
attributes with varying spatial extent, we propose a mask expansion strategy
using keypoint-conditioned image generation for training. In addition, we
further augment the attribute and portrait images with spatial and
appearance-level transformations to improve robustness to positional
misalignment between them. These strategies allow the model to effectively
generalize across diverse attributes and in-the-wild reference combinations,
despite being trained without explicit triplet supervision. Durian achieves
state-of-the-art performance on portrait animation with attribute transfer, and
notably, its dual reference design enables multi-attribute composition in a
single generation pass without additional training.
Durian uses dual reference networks and a diffusion model to generate high-fidelity portrait animations with attribute transfer from a reference image to a target portrait in a zero-shot manner.
dual reference networks denoising process diffusion model self-reconstruction formulation mask expansion strategy keypoint-conditioned image generation spatial and appearance-level transformations multi-attribute composition