Context Diffusion: In-Context Aware Image Generation
We propose Context Diffusion, a diffusion-based framework that enables image
generation models to learn from visual examples presented in context. Recent
work tackles such in-context learning for image generation, where a query image
is provided alongside context examples and text prompts. However, the quality
and fidelity of the generated images deteriorate when the prompt is not
present, demonstrating that these models are unable to truly learn from the
visual context. To address this, we propose a novel framework that separates
the encoding of the visual context and preserving the structure of the query
images. This results in the ability to learn from the visual context and text
prompts, but also from either one of them. Furthermore, we enable our model to
handle few-shot settings, to effectively address diverse in-context learning
scenarios. Our experiments and user study demonstrate that Context Diffusion
excels in both in-domain and out-of-domain tasks, resulting in an overall
enhancement in image quality and fidelity compared to counterpart models.
657131be4c829fb8f5e415eb
Ivona Najdenkoska
claimed_verified
2023-12-07T15:29:52.430Z
6571e0ab487dfeeb4171ca4d
/avatars/2ab286ad1f502252d71bfed8a54f4cbf.svg
Ivona Najdenkoska
user
ivonajdenkoska
657131be4c829fb8f5e415ec
Animesh Sinha
admin_assigned
2023-12-07T12:05:19.952Z
63772d28e069bdcdbb8e4a29
/avatars/be71b00e009f2093a3ff79f07815e68e.svg
Animesh Sinha
user
animeshsinha
657131be4c829fb8f5e415ed
Abhimanyu Dubey
admin_assigned
2023-12-07T12:05:27.380Z
6376f5a98272ecd8ddacacf8
/avatars/c57d5de9a90ec12b7f617327b1ac27fd.svg
Abhimanyu Dubey
user
dubeya
657131be4c829fb8f5e415ee
Dhruv Mahajan
admin_assigned
2023-12-07T12:05:33.968Z
64358a51f81a16e7436330f7
/avatars/71c350836e5d025b4c7b9f99f72c2e21.svg
Dhruv Mahajan
user
Dhruv2009
657131be4c829fb8f5e415ef
Vignesh Ramanathan
657131be4c829fb8f5e415f0
Filip Radenovic
admin_assigned
2023-12-07T12:05:56.935Z
6515dccbc979eb9ca72dcc8c
/avatars/b55101dc2b0052c36f0b2cbdea9473e7.svg
Filip Radenovic
user
filipradenovic
Context Diffusion: In-Context Aware Image Generation
We propose Context Diffusion, a diffusion-based framework that enables image
generation models to learn from visual examples presented in context. Recent
work tackles such in-context learning for image generation, where a query image
is provided alongside context examples and text prompts. However, the quality
and fidelity of the generated images deteriorate when the prompt is not
present, demonstrating that these models are unable to truly learn from the
visual context. To address this, we propose a novel framework that separates
the encoding of the visual context and preserving the structure of the query
images. This results in the ability to learn from the visual context and text
prompts, but also from either one of them. Furthermore, we enable our model to
handle few-shot settings, to effectively address diverse in-context learning
scenarios. Our experiments and user study demonstrate that Context Diffusion
excels in both in-domain and out-of-domain tasks, resulting in an overall
enhancement in image quality and fidelity compared to counterpart models.
Context Diffusion enhances in-context image generation by separately encoding visual context and preserving query image structure, improving quality and fidelity across different scenarios.
diffusion-based framework visual context query images in-context learning few-shot settings image quality image fidelity