VLM-Guided Adaptive Negative Prompting for Creative Generation
Creative generation is the synthesis of new, surprising, and valuable samples
that reflect user intent yet cannot be envisioned in advance. This task aims to
extend human imagination, enabling the discovery of visual concepts that exist
in the unexplored spaces between familiar domains. While text-to-image
diffusion models excel at rendering photorealistic scenes that faithfully match
user prompts, they still struggle to generate genuinely novel content. Existing
approaches to enhance generative creativity either rely on interpolation of
image features, which restricts exploration to predefined categories, or
require time-intensive procedures such as embedding optimization or model
fine-tuning. We propose VLM-Guided Adaptive Negative-Prompting, a
training-free, inference-time method that promotes creative image generation
while preserving the validity of the generated object. Our approach utilizes a
vision-language model (VLM) that analyzes intermediate outputs of the
generation process and adaptively steers it away from conventional visual
concepts, encouraging the emergence of novel and surprising outputs. We
evaluate creativity through both novelty and validity, using statistical
metrics in the CLIP embedding space. Through extensive experiments, we show
consistent gains in creative novelty with negligible computational overhead.
Moreover, unlike existing methods that primarily generate single objects, our
approach extends to complex scenarios, such as generating coherent sets of
creative objects and preserving creativity within elaborate compositional
prompts. Our method integrates seamlessly into existing diffusion pipelines,
offering a practical route to producing creative outputs that venture beyond
the constraints of textual descriptions.
68ee4c979b77b5223f6661fc
Shelly Golan
claimed_verified
2025-10-14T14:23:57.647Z
64646a88a5af935cfe741d3b
https://cdn-avatars.huggingface.co/v1/production/uploads/64646a88a5af935cfe741d3b/DEmHne74aRyd3XjHz82_M.png
shelly golan
user
shellygolan
68ee4c979b77b5223f6661fd
Yotam Nitzan
68ee4c979b77b5223f6661fe
Zongze Wu
68ee4c979b77b5223f6661ff
Or Patashnik
claimed_verified
2025-10-16T06:42:02.013Z
62853516e483e0d37b354ce1
https://cdn-avatars.huggingface.co/v1/production/uploads/62853516e483e0d37b354ce1/t5Tyd3E07w26B9Z3XpZWI.jpeg
Or Patashnik
user
orpatashnik
VLM-Guided Adaptive Negative Prompting for Creative Generation
Creative generation is the synthesis of new, surprising, and valuable samples
that reflect user intent yet cannot be envisioned in advance. This task aims to
extend human imagination, enabling the discovery of visual concepts that exist
in the unexplored spaces between familiar domains. While text-to-image
diffusion models excel at rendering photorealistic scenes that faithfully match
user prompts, they still struggle to generate genuinely novel content. Existing
approaches to enhance generative creativity either rely on interpolation of
image features, which restricts exploration to predefined categories, or
require time-intensive procedures such as embedding optimization or model
fine-tuning. We propose VLM-Guided Adaptive Negative-Prompting, a
training-free, inference-time method that promotes creative image generation
while preserving the validity of the generated object. Our approach utilizes a
vision-language model (VLM) that analyzes intermediate outputs of the
generation process and adaptively steers it away from conventional visual
concepts, encouraging the emergence of novel and surprising outputs. We
evaluate creativity through both novelty and validity, using statistical
metrics in the CLIP embedding space. Through extensive experiments, we show
consistent gains in creative novelty with negligible computational overhead.
Moreover, unlike existing methods that primarily generate single objects, our
approach extends to complex scenarios, such as generating coherent sets of
creative objects and preserving creativity within elaborate compositional
prompts. Our method integrates seamlessly into existing diffusion pipelines,
offering a practical route to producing creative outputs that venture beyond
the constraints of textual descriptions.
A method using vision-language models to enhance creative image generation by adaptively steering away from conventional concepts, improving novelty with minimal computational cost.
text-to-image diffusion models VLM-Guided Adaptive Negative-Prompting vision-language model CLIP embedding space creative image generation intermediate outputs novelty validity diffusion pipelines