Video relighting aims to replace the background in videos while correspondingly adjusting the lighting in the foreground with harmonious blending. Lumen preserves the original properties of the foreground (e.g., albedo) and propagates consistent relighting across temporal frames. It is trained on a large-scale dataset featuring a mixture of realistic and synthetic videos, utilizing a domain-aware adapter to decouple the learning of relighting and domain appearance distribution.
Jianshu Zeng, Yuxuan Liu, Yutong Feng, Chenxuan Miao, Zixiang Gao, Jiwang Qu, Jianzhang Zhang, Bin Wang, Kun Yuan.
This repository contains the weights of
Lumen. For more instructions about how to use the model, please refer to the
official GitHub repository.
1conda create -n lumen python=3.10 -y
2conda activate lumen
3pip install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124
4pip install -r requirements.txt
After downloading the model weights and necessary base models (Wan2.1-Fun), you can run inference or the Gradio app:
1# Run text-to-video inference
2python infer_t2v.py
3
4# Launch the Gradio demo
5python app_lumen.py
1@article{zeng2025lumen,
2 title={Lumen: Consistent Video Relighting and Harmonious Background Replacement with Video Generative Models},
3 author={Zeng, Jianshu and Liu, Yuxuan and Feng, Yutong and Miao, Chenxuan and Gao, Zixiang and Qu, Jiwang and Zhang, Jianzhang and Wang, Bin and Yuan, Kun},
4 journal={arXiv preprint arXiv:2508.12945},
5 year={2025},
6 url={https://arxiv.org/abs/2508.12945},
7}
We would like to thank the contributors to
DiffSynth-Studio,
VideoX-Fun, and the
Wan2.1 team for their open research and exploration.
