IllumiCraft is a controllable video generation framework that jointly models scene geometry and illumination dynamics. By leveraging foreground appearance, HDR lighting maps, and 3D point tracks, IllumiCraft enables temporally consistent video generation with controllable motion and relighting effects.
Please refer to the official GitHub repository for installation, dataset preparation, training, and inference instructions:
1@article{lin2026illumicraft,
2 title={IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation},
3 author={Lin, Yuanze and Chen, Yi-Wen and Tsai, Yi-Hsuan and Clark, Ronald and Yang, Ming-Hsuan},
4 journal={Advances in Neural Information Processing Systems},
5 volume={38},
6 pages={27798--27829},
7 year={2026}
8}
