UniLumos: Fast and Unified Image and Video Relighting with Physics-Plausible Feedback
Pengwei Liu, Hangjie Yuan, Bo Dong, Jiazheng Xing, Jinwang Wang, Rui Zhao, Weihua Chen, Fan Wang
Image and video relighting remains a challenging task, where controlling illumination under ensures physical consistency, and efficiency under diverse lighting conditions.However, existing relighting methods lack explicit physical grounding, often resulting in perceptually implausible illumination under diverse lighting conditions, such as overexposed highlights or inconsistent shadow placement.In addition, similar to general-purpose generative models, their reliance on multi-iterative sampling leads to slow inference. This is particularly problematic considering the strong conditioning in relighting tasks, which should intuitively enable much faster generation, especially for real-time applications.To address these challenges, we propose UniLumos, a unified framework for fast and physically plausible relighting of both images and videos. First, we introduce LumosData, a lightweight pipeline for constructing diverse relighting pairs from the illumination augmentations of real-world data. Leveraging the strong conditioning provided by LumosData, UniLumos extends a flow-matching backbone with path consistency learning to enable fast inference without multi-step sampling.Second, inspired by the inherent geometric consistency in dense estimation, we introduce a physics-plausible feedback mechanism that supervises the latent diffusion process through depth and normal alignment with the labeled references, encouraging physically realistic illumination.Experiments demonstrate that UniLumos achieves high-quality and physically plausible relighting with significantly faster inference for both image and video relighting.