LightLab: Controlling Light Sources in Images with Diffusion Models
We present a simple, yet effective diffusion-based method for fine-grained,
parametric control over light sources in an image. Existing relighting methods
either rely on multiple input views to perform inverse rendering at inference
time, or fail to provide explicit control over light changes. Our method
fine-tunes a diffusion model on a small set of real raw photograph pairs,
supplemented by synthetically rendered images at scale, to elicit its
photorealistic prior for relighting. We leverage the linearity of light to
synthesize image pairs depicting controlled light changes of either a target
light source or ambient illumination. Using this data and an appropriate
fine-tuning scheme, we train a model for precise illumination changes with
explicit control over light intensity and color. Lastly, we show how our method
can achieve compelling light editing results, and outperforms existing methods
based on user preference.
68262ba929fe8333cdfec69c
Nadav Magar
extracted_pending
2025-05-15T18:00:14.378Z
6698f6ee004e418e71fbbdef
/avatars/3c2d521c5ece4fc5a0a483a3238bf659.svg
Nadav Magar
user
NadMag
68262ba929fe8333cdfec69d
Amir Hertz
68262ba929fe8333cdfec69e
Eric Tabellion
68262ba929fe8333cdfec69f
Yael Pritch
68262ba929fe8333cdfec6a0
Alex Rav-Acha
68262ba929fe8333cdfec6a1
Ariel Shamir
68262ba929fe8333cdfec6a2
Yedid Hoshen
LightLab: Controlling Light Sources in Images with Diffusion Models
We present a simple, yet effective diffusion-based method for fine-grained,
parametric control over light sources in an image. Existing relighting methods
either rely on multiple input views to perform inverse rendering at inference
time, or fail to provide explicit control over light changes. Our method
fine-tunes a diffusion model on a small set of real raw photograph pairs,
supplemented by synthetically rendered images at scale, to elicit its
photorealistic prior for relighting. We leverage the linearity of light to
synthesize image pairs depicting controlled light changes of either a target
light source or ambient illumination. Using this data and an appropriate
fine-tuning scheme, we train a model for precise illumination changes with
explicit control over light intensity and color. Lastly, we show how our method
can achieve compelling light editing results, and outperforms existing methods
based on user preference.
A diffusion-based method fine-tuned on real and synthetic image pairs provides precise control over light sources and ambient illumination in images, offering more effective relighting than existing methods.
diffusion-based method fine-tuning diffusion model relighting inverse rendering photorealistic prior linearity of light light intensity light color light editing