The Cow of Rembrandt - Analyzing Artistic Prompt Interpretation in
  Text-to-Image Models
Text-to-image diffusion models have demonstrated remarkable capabilities in
generating artistic content by learning from billions of images, including
popular artworks. However, the fundamental question of how these models
internally represent concepts, such as content and style in paintings, remains
unexplored. Traditional computer vision assumes content and style are
orthogonal, but diffusion models receive no explicit guidance about this
distinction during training. In this work, we investigate how transformer-based
text-to-image diffusion models encode content and style concepts when
generating artworks. We leverage cross-attention heatmaps to attribute pixels
in generated images to specific prompt tokens, enabling us to isolate image
regions influenced by content-describing versus style-describing tokens. Our
findings reveal that diffusion models demonstrate varying degrees of
content-style separation depending on the specific artistic prompt and style
requested. In many cases, content tokens primarily influence object-related
regions while style tokens affect background and texture areas, suggesting an
emergent understanding of the content-style distinction. These insights
contribute to our understanding of how large-scale generative models internally
represent complex artistic concepts without explicit supervision. We share the
code and dataset, together with an exploratory tool for visualizing attention
maps at https://github.com/umilISLab/artistic-prompt-interpretation.
68905b400a411b3b8d28d6f2
Alfio Ferrara
68905b400a411b3b8d28d6f3
Sergio Picascia
claimed_verified
2025-08-06T19:25:00.992Z
6458e5e8c16ecb4815de8aac
/avatars/c33c1a67960cdbbc8a970d751dff07af.svg
Sergio Picascia
user
sergiopicascia
68905b400a411b3b8d28d6f4
Elisabetta Rocchetti
The Cow of Rembrandt - Analyzing Artistic Prompt Interpretation in
  Text-to-Image Models
Text-to-image diffusion models have demonstrated remarkable capabilities in
generating artistic content by learning from billions of images, including
popular artworks. However, the fundamental question of how these models
internally represent concepts, such as content and style in paintings, remains
unexplored. Traditional computer vision assumes content and style are
orthogonal, but diffusion models receive no explicit guidance about this
distinction during training. In this work, we investigate how transformer-based
text-to-image diffusion models encode content and style concepts when
generating artworks. We leverage cross-attention heatmaps to attribute pixels
in generated images to specific prompt tokens, enabling us to isolate image
regions influenced by content-describing versus style-describing tokens. Our
findings reveal that diffusion models demonstrate varying degrees of
content-style separation depending on the specific artistic prompt and style
requested. In many cases, content tokens primarily influence object-related
regions while style tokens affect background and texture areas, suggesting an
emergent understanding of the content-style distinction. These insights
contribute to our understanding of how large-scale generative models internally
represent complex artistic concepts without explicit supervision. We share the
code and dataset, together with an exploratory tool for visualizing attention
maps at https://github.com/umilISLab/artistic-prompt-interpretation.
Transformer-based text-to-image diffusion models show varying degrees of content-style separation in generated artworks, as revealed by cross-attention heatmaps.
text-to-image diffusion models transformer-based cross-attention heatmaps content-style separation artistic prompts attention maps