(*Equal contribution, †This work was conducted during the author's internship at Kling, ‡Corresponding author)
📖 Introduction
TL;DR: We propose StereoPilot, an efficient feed-forward architecture that leverages pretrained video diffusion transformers to directly synthesize novel views, overcoming the limitations of Depth-Warp-Inpaint methods without iterative denoising. With a domain switcher and cycle consistency loss, it enables robust multi-format stereo conversion. We also introduce UniStereo, the first large-scale unified dataset featuring both parallel and converged stereo formats.
Video file (.mp4): Monocular video, 81 frames, 832×480 resolution, 16fps
Prompt file (.txt): Text description of the video content (same name as video)
Example (you can try the cases in the sample/ folder):
sample/
├── my_video.mp4
└── my_video.txt
Running Inference
Basic usage:
bash
1# Edit toml/infer.toml to customize model paths. If you followed the above steps, there is no need to change2python sample.py \3 --config toml/infer.toml \4 --input /path/to/input_video.mp4 \5 --output_folder /path/to/output \6 --device cuda:0
Using the example script:
bash sample.sh
Generate Stereo Visualization
After inference, you can generate Side-by-Side (SBS) and Red-Cyan anaglyph stereo videos for visualization:
If you find our work helpful, please consider citing:
bibtex
1@misc{shen2025stereopilot,
2 title={StereoPilot: Learning Unified and Efficient Stereo Conversion via Generative Priors},
3 author={Shen, Guibao and Du, Yihua and Ge, Wenhang and He, Jing and Chang, Chirui and Zhou, Donghao and Yang, Zhen and Wang, Luozhou and Tao, Xin and Chen, Ying-Cong},
4 year={2025},
5 eprint={2512.16915},
6 archivePrefix={arXiv},
7 primaryClass={cs.CV},
8 url={https://arxiv.org/abs/2512.16915},
9}
