Artist: Aesthetically Controllable Text-Driven Stylization without
  Training
Diffusion models entangle content and style generation during the denoising
process, leading to undesired content modification when directly applied to
stylization tasks. Existing methods struggle to effectively control the
diffusion model to meet the aesthetic-level requirements for stylization. In
this paper, we introduce Artist, a training-free approach that
aesthetically controls the content and style generation of a pretrained
diffusion model for text-driven stylization. Our key insight is to disentangle
the denoising of content and style into separate diffusion processes while
sharing information between them. We propose simple yet effective content and
style control methods that suppress style-irrelevant content generation,
resulting in harmonious stylization results. Extensive experiments demonstrate
that our method excels at achieving aesthetic-level stylization requirements,
preserving intricate details in the content image and aligning well with the
style prompt. Furthermore, we showcase the highly controllability of the
stylization strength from various perspectives. Code will be released, project
home page: https://DiffusionArtist.github.io
669f412fab5548b23ea16ae2
Ruixiang Jiang
admin_assigned
2024-07-23T12:23:47.673Z
6351382f40dffad651ef3fbd
/avatars/3ac2de7c49086bb37cc4f4bd29ed72f2.svg
JIANG
user
Ruixiang
669f412fab5548b23ea16ae3
Changwen Chen
Artist: Aesthetically Controllable Text-Driven Stylization without
  Training
Diffusion models entangle content and style generation during the denoising
process, leading to undesired content modification when directly applied to
stylization tasks. Existing methods struggle to effectively control the
diffusion model to meet the aesthetic-level requirements for stylization. In
this paper, we introduce Artist, a training-free approach that
aesthetically controls the content and style generation of a pretrained
diffusion model for text-driven stylization. Our key insight is to disentangle
the denoising of content and style into separate diffusion processes while
sharing information between them. We propose simple yet effective content and
style control methods that suppress style-irrelevant content generation,
resulting in harmonious stylization results. Extensive experiments demonstrate
that our method excels at achieving aesthetic-level stylization requirements,
preserving intricate details in the content image and aligning well with the
style prompt. Furthermore, we showcase the highly controllability of the
stylization strength from various perspectives. Code will be released, project
home page: https://DiffusionArtist.github.io
Artist is a training-free method to control a pretrained diffusion model for text-driven stylization, achieving aesthetic results by separating and sharing information between content and style denoising processes.
diffusion models denoising process text-driven stylization aesthetic-level requirements harmonious stylization stylization strength