WildRayZer
This repository hosts the checkpoint of
WildRayZer: Self-supervised Large View Synthesis in Dynamic Environments
(CVPR 2026, Highlight).
Model summary
WildRayZer is a self-supervised feed-forward framework for novel view synthesis
(NVS) in
dynamic in-the-wild videos where both the camera and scene objects
move. It extends the static NVS model
RayZer
to dynamic environments by adding:
- a learned motion mask estimator that flags dynamic regions per input
view, trained by distilling pseudo-masks from the residual between a static
renderer and the observed frames (DINOv3 + SSIM + co-segmentation + GrabCut);
- a masked 3D scene encoder that replaces dynamic image tokens with a
learnable noise embedding before scene aggregation (MAE-style token masking).
All supervision is self-supervised — no ground-truth depth, camera poses,
or motion masks are used. Given a set of unposed, uncalibrated dynamic images,
the model predicts camera parameters and motion masks and renders novel static
views in a single feed-forward pass.
This checkpoint
| Property | Value |
|---|
| File | wildrayzer_2view.pt (3.9 GB, fp32 state_dict) |
| Input resolution | 256 × 256 |
| Input / target views | 2 input → 6 target |
| Base dataset | Dynamic-RE10K (train split) + RealEstate10K (static mix-in) |
| Backbone | RayZer (28 transformer layers) + DINOv3 ViT-7B features |
| Framework | PyTorch ≥ 2.1, xFormers, transformers |
The K=2 configuration matches the sparse-view setting used in the paper's
main D-RE10K and D-RE10K-iPhone benchmarks. 3- and 4-input-view variants can
be reproduced by retraining with the same pipeline — see
training details.
How to use
Download the checkpoint and run the reference demo:
1from huggingface_hub import hf_hub_download
2
3ckpt_path = hf_hub_download(
4 repo_id="uva-cv-lab/wildrayzer-2view",
5 filename="wildrayzer_2view.pt",
6)
7# Pass ckpt_path to the WildRayZerDemo class or to inference.py
8# via --config configs/wildrayzer_inference.yaml.
The full inference pipeline, Gradio demo, and training code live in the
companion repo. A ready-to-deploy
Space layout is provided under
demo/ in that repo.
Hardware requirements: CUDA GPU with ≥ 40 GB VRAM (the motion-mask
predictor fuses DINOv3 ViT-7B patch features with image/Plücker tokens at
inference time — this 7B backbone is a hard dependency, not optional). The author will soon provide an alternative.
Citation
1@inproceedings{chen2026wildrayzer,
2 title = {WildRayZer: Self-supervised Large View Synthesis in Dynamic Environments},
3 author = {Chen, Xuweiyi and Zhou, Wentao and Cheng, Zezhou},
4 booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
5 note = {Highlight},
6 year = {2026},
7}
License
Released under CC BY-NC 4.0 — free for research and non-commercial use,
attribution required. For commercial licensing, contact the authors.
Acknowledgements
This work was supported by the MathWorks Research Gift, Adobe Research Gift, the University of Virginia
Research Computing and Data Analytics Center, the AMD AI & HPC Cluster Program,
the ACCESS program, and the NAIRR Pilot. Computation was run on the Anvil
supercomputer (NSF OAC-2005632) at Purdue and on Delta / DeltaAI (NSF
OAC-2005572).
