SpotLight: Shadow-Guided Object Relighting via Diffusion
Recent work has shown that diffusion models can be used as powerful neural
rendering engines that can be leveraged for inserting virtual objects into
images. Unlike typical physics-based renderers, however, neural rendering
engines are limited by the lack of manual control over the lighting setup,
which is often essential for improving or personalizing the desired image
outcome. In this paper, we show that precise lighting control can be achieved
for object relighting simply by specifying the desired shadows of the object.
Rather surprisingly, we show that injecting only the shadow of the object into
a pre-trained diffusion-based neural renderer enables it to accurately shade
the object according to the desired light position, while properly harmonizing
the object (and its shadow) within the target background image. Our method,
SpotLight, leverages existing neural rendering approaches and achieves
controllable relighting results with no additional training. Specifically, we
demonstrate its use with two neural renderers from the recent literature. We
show that SpotLight achieves superior object compositing results, both
quantitatively and perceptually, as confirmed by a user study, outperforming
existing diffusion-based models specifically designed for relighting.
674e0997e7861d5a0277b79f
Frédéric Fortier-Chouinard
674e0997e7861d5a0277b7a0
Zitian Zhang
claimed_verified
2024-12-02T22:07:18.112Z
655dbda6668b64adf1495528
/avatars/dbf6072dcf0905873289d78209a92f87.svg
Zitian Zhang
user
zzt76
674e0997e7861d5a0277b7a1
Louis-Etienne Messier
674e0997e7861d5a0277b7a2
Mathieu Garon
674e0997e7861d5a0277b7a3
Anand Bhattad
claimed_verified
2025-06-29T10:15:02.956Z
6303d5924ec2dfa82a5627de
/avatars/1e433b7fb9d30b9f42dec487d658b1e3.svg
Anand Bhattad
user
anandbhattad
674e0997e7861d5a0277b7a4
Jean-François Lalonde
SpotLight: Shadow-Guided Object Relighting via Diffusion
Recent work has shown that diffusion models can be used as powerful neural
rendering engines that can be leveraged for inserting virtual objects into
images. Unlike typical physics-based renderers, however, neural rendering
engines are limited by the lack of manual control over the lighting setup,
which is often essential for improving or personalizing the desired image
outcome. In this paper, we show that precise lighting control can be achieved
for object relighting simply by specifying the desired shadows of the object.
Rather surprisingly, we show that injecting only the shadow of the object into
a pre-trained diffusion-based neural renderer enables it to accurately shade
the object according to the desired light position, while properly harmonizing
the object (and its shadow) within the target background image. Our method,
SpotLight, leverages existing neural rendering approaches and achieves
controllable relighting results with no additional training. Specifically, we
demonstrate its use with two neural renderers from the recent literature. We
show that SpotLight achieves superior object compositing results, both
quantitatively and perceptually, as confirmed by a user study, outperforming
existing diffusion-based models specifically designed for relighting.
SpotLight enables controllable relighting in neural rendering by specifying object shadows, outperforming existing models without additional training.
diffusion models neural renderers object relighting shadows SpotLight object compositing user study