SCoPE adds camera sightlines as positional coordinates to a pretrained video diffusion
transformer. Given a first frame, a text prompt, and a camera trajectory, it generates a video
that follows the requested camera motion while preserving the original image-to-video prior. This
repository is a self-contained release for Wan2.2-I2V-A14B: it contains everything required for
inference, so a separate Wan2.2 checkpoint download is not needed.
The checkpoint is approximately 67 GB. Keep both the checkpoint and the Hugging Face cache on local
storage.
Usage
Install the SCoPE code. The released weights were trained and evaluated with PyTorch 2.9.1
(CUDA 12.8); because changing the PyTorch version can change the numerical output, we recommend
reproducing this exact environment with uv:
1python inference.py \2 --model_path checkpoints/SCoPE \3 --input_image path/to/first_frame.png \4 --prompt "A person walks along a misty forest trail."\5 --camera_path path/to/camera_poses.npy \6 --x_fov 1.11847\7 --output_path outputs/custom.mp4
Camera poses use OpenCV camera-to-world coordinates and must have shape [81, 3, 4] or
[81, 4, 4]. x_fov is the horizontal field of view in radians; pinhole cameras use xi=0. See
the GitHub repository for the full documentation, options,
and demos.
Training data
SCoPE is trained with RealEstate10K, DL3DV, PanShot, and OmniWorld. The datasets use a common
camera protocol: poses are expressed relative to the first camera and translation is normalized
with per-clip near depth, while absolute scale is handled inside the model by a learned scale gate.
Users are responsible for following the licenses and terms of the corresponding datasets.
Intended use and limitations
This model is intended for research on image-to-video generation and controllable camera motion.
It inherits the visual capabilities, biases, safety limitations, and computational requirements of
Wan2.2. Results may degrade for inaccurate camera poses or intrinsics, trajectories far outside the
training distribution, large occlusions, or unusually fast camera motion.
Citation
bibtex
1@article{yin2026scope,
2 title={SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformers},
3 author={Yin, Minghao and Lu, Jiahao and Hu, Wenbo and Zhao, Wang and Shan, Ying and Han, Kai},
4 year={2026}
5}
Acknowledgements
SCoPE is built on Wan2.2 and
DiffSynth-Studio. We thank the authors and
contributors of these projects.
