VR-360-Outpaint 1.0 — LTX-2.3 IC-LoRA
THE 1.0 IS FINALLY OUT!
360° equirectangular video outpainting. Give it a flat (converted to inverse gnomonic by my nodes) video crop on a black
equirectangular canvas, and it fills the unknown regions to produce a full 360° equirectangular
video suitable for VR / immersive playback.
Built on Lightricks/LTX-2.3-22B via IC-LoRA (video-to-video conditioning). This is the production "v1.0" run — a fully retrained IC-LoRA, multi-reference augmentation, and mixed-resolution buckets.
How it works: the inverse gnomonic projection trick
The core insight behind VR-360-Outpaint — first developed in April 2026 by yours truly — is that a video diffusion
model can struggle with equirectangular geometry. Asking it to "extend this flat crop into 360°" without geometric guidance is a near-impossible task: the model must simultaneously understand both the
spherical projection mapping and the outpainting task.
The solution that I came up with was to take the rectilinear input video and apply an inverse gnomonic projection,
compositing it onto the equirectangular canvas at the correct angular position and field of view. This gives the
model an explicit geometric nudge — it can see exactly where and how the known pixels sit on the sphere, and what
the local rectilinear perspective looks like in spherical space. The outpainting task then becomes: 'given this
correctly mapped window onto the sphere, complete the rest of the equirectangular frame in a way that is consistent with it.'
This geometric conditioning is applied identically in
both training and inference — every training
pair was a 360° video paired with a gnomonic crop of a specific FOV and aspect ratio
on the canvas, and the model learned to recover the full equirectangular frame from that partial view.
At inference time, the
ComfyUI-VR-Outpaint-Tools
node pack performs the same projection from your flat footage, now also including automatic FOV estimation via
the
GeoCalib node.
Without this geometric scaffolding, the model would have to discover spherical projection rules from
its core training — although LTX2.3 obviously has some equirectangular material in its core training data, this still presents a vastly harder learning problem. The gnomonic projection provides the inductive bias that
makes VR outpainting tractable.
Quick start
- Base model:
Lightricks/LTX-2.3-22B (dev safetensors)
- Load this LoRA on top via the LTX-2
video_to_video pipeline (ComfyUI recommended)
- Trigger word:
equirectangular
- Caption (the only one the model knows):
360 degree equirectangular panorama, high quality, detailed
- Resolution: 1920×960, 121 frames @ 24 fps
- Workflow: Use the included
Burgstall-VR-Outpaint.json in ComfyUI
- Companion tools: ComfyUI-VR-Outpaint-Tools
provides the gnomonic projection nodes and the GeoCalib FOV estimation node for automatic
field-of-view detection from your flat footage — no manual FOV entry needed.
Examples
Each example shows the flat input video, a slow gnomonic pan across the outpainted 360° result
(a virtual camera sweeping through the scene), and the raw equirectangular output.
| Flat input | Pan across the result | Raw equirectangular output |
| | |
| | |
| | |
Which checkpoint?
Three checkpoints are provided (.safetensors, ~1.3 GB each). They all produce valid equirectangular
output; pick based on your priorities:
| Checkpoint | Step | Character |
|---|
360vroutpaint_v2_step07000.safetensors | 7000 | Recommended. Best balance of equirectangular consistency and scene variety |
360vroutpaint_v2_step05000.safetensors | 5000 | Slightly more creative outpainting, but the 360° wrap can have a more prominent seam |
360vroutpaint_v2_step09000.safetensors | 9000 | Tightest equirectangular consistency, but tends toward more uniform / monotonic scenery |
All three are fully converged. The
differences are subtle; if unsure, start with step 7000.
Training
| Parameter | Value |
|---|
| Base model | LTX-2.3-22B (dev) |
| Strategy | IC-LoRA (video_to_video, first_frame_conditioning_p: 0.0) |
| LoRA rank / alpha | 128 / 128 |
| Target modules | Self-attn (q, k, v, out) + cross-attn (q, k, v, out) + FFN (ff.net.0.proj, ff.net.2) |
| Trainable params | 654M |
| Resolution | Mixed buckets: 1024×512×121 + 1920×960×49 |
| Frame rate | 24 fps |
| Optimizer | Prodigy (D-Adaptation), lr=1.0, decoupled weight decay, constant schedule |
| Precision | bf16 (weights + Gemma text encoder), gradient checkpointing ON |
| Batch / accum | 1 / 1 |
| Steps trained | 9000 (converged by ~3000, stopped on plateau) |
| Hardware | 1× RTX PRO 6000 Blackwell (96 GB), compute sponsored by Lightricks |
| Caption | 360 degree equirectangular panorama, high quality, detailed |
Dataset
The model was trained on 13,281 pairs, sampled from a master dataset of 53,124 augmented
video pairs built from 4,427 unique 360° clips sourced from 51 CC-BY YouTube channels.
Each of the 4,427 clips was rendered into 12 reference variants — the combination of:
- 4 fields of view: 70°, 90°, 110°, 130°
- 3 aspect ratios: 1.33 (4:3), 1.78 (16:9), 2.39 (cinemascope)
From these 12 variants per clip, 3 distinct FOVs were assigned per target (globally balanced across
all 12 FOV×aspect cells, ~1,107 pairs each), and the trainer randomly sampled one of the 3 at each step.
A per-clip random yaw shift (0–1920 px) exploits equirectangular rotational symmetry, effectively
multiplying each scene into many independent viewpoints and keeping operators / tripods off a fixed azimuth.
All clips in the master dataset were normalized to 1920×960 at 24 fps, 121 frames (~5 seconds).
The training dataset is private and not distributed with these weights.
Validation
153 held-out clips from 6 diverse non-CC sources (urban day/night, indoor, nature, action, aurora).
Eval-only — never trained on. Validation is sample-generation only (no held-out val-loss exists).
Convergence evidence
Loss is a poor signal for IC-LoRA. I tracked three independent metrics; all agree the model converged
by step 3000 and plateaued through 9000:
1. 360° seam continuity (seam_ratio — wrap-edge MSE ÷ mean adjacent-column MSE, lower is better):
| Step | Seam ratio |
|---|
| 1000 | 59.7 |
| 2000 | 33.4 |
| 3000 | 6.1 ← converged |
| 5000 | 9.3 |
| 7000 | 5.1 |
| 9000 | 5.6 |
The wrap seam dropped from ~60 to ~6 in 3000 steps and stayed
in the 5–9 band (noise floor) through 9000. The residual ~5–6 is a data/config limit, not fixable
by more steps.
2. Weight delta (relative ‖Wₙ − Wₙ₋₅₀₀‖):
- Early: 0.15–0.17 (high, noisy learning)
- Step 5000: 0.137
- Step 7500+: 0.126–0.135 (stable plateau — no continued learning)
3. Visual inspection: Side-by-side comparisons at steps 5000, 7000, and 9000 are
visually near-identical for most scenes, with mostly just subtle sharpening differences as well as negligible differences in seam.
Sweet spot
The model was tuned on and performs best with:
- Content: Semi-static establishing shots — cityscapes, urban scenes, landscapes
- FOV: ~90–110° horizontal (the training distribution center)
- Aspect ratio: 1.78–2.39 (standard widescreen to cinemascope)
- Camera motion: Slow pans, gentle dolly, static tripod
It will generalize beyond these conditions — that's why I trained across 4 FOVs and 3 aspect
ratios — but quality degrades gracefully as you move further from the training distribution.
Limitations
- Top & bottom caps (zenith / nadir of the sphere) are the hardest regions; the model has less
signal there. Avoid critical detail near the poles.
- Busy motion (fast cuts, whip pans, handheld shake) easily break the wrap seam.
- Prompt adherence — conditioning is dominated by the reference video, not text. The base caption / prompt is good to keep there, but you can describe the elements that you want there to be in the outpainted area to guide it. It will outpaint with the base prompt only, too.
- Narrow domain — trained on outdoor / architectural scenes. Interiors, crowds, close-ups,
and organic subjects might work less reliably, although these draw strongly from the model's base training.
Companion tools
- ComfyUI-VR-Outpaint-Tools —
A ComfyUI node pack providing:
- Gnomonic projection nodes for converting flat footage into the masked equirectangular
reference this LoRA expects (FOV selection, projection, canvas compositing)
- GeoCalib FOV estimation node for automatic field-of-view detection from your footage —
no manual measurement needed
- Formerly named ComfyUI-EquirectProjector; renamed after expanding scope beyond the original idea of just forward gnomonic projection.
Burgstall-VR-Outpaint.json (included in this repo) — End-to-end ComfyUI workflow wiring the
tools → LTX-2.3 + this LoRA → output.
⚠️ Repo re-star request: The node pack was renamed from ComfyUI-EquirectProjector to
ComfyUI-VR-Outpaint-Tools, and in the process I accidentally turned the repo private for a
moment — which resets the star count to zero. If you starred it before, I'd really appreciate
it if you could drop by and re-star it if you appreciate this work. Every one helps! ⭐
What's next
I am currently in the process of developing a seamless equirectangular video generation method. Stay tuned.
Acknowledgements
- Compute sponsorship by Lightricks — the LTX team graciously
provided the GPU hardware for this training run. Special thanks to Ofir Bibi for technical
guidance and advice throughout the project.
- Initial PoC training done at ADOS Paris 2026 with collaborators Cseti, NebSH, and S4f3ty_Marc. Thank you for your collaboration and support.
- IC-LoRA training methodologies and multi-reference dataset design advised by oumoumad.
- Built on the Lightricks LTX-2.3 foundation model.
Training data attribution
The model was trained on 360° video clips from the following YouTube creators, all shared under
Creative Commons Attribution (CC-BY) license. Thank you to each of them for making their work
available for reuse.
A standalone copy of these credits is available in
ATTRIBUTIONS.md.
All clips were obtained under CC-BY as indicated by YouTube at the time of download. License proof
(.info.json sidecar files) is archived.
License
This VR-Outpaint IC-LoRA is distributed under the
Burgstall LoRA License. Non-commercial use of Burgstall's contribution is permitted subject to those terms. Commercial use of Burgstall's contribution requires a separate commercial license.
Commercial licensing inquiries:
howdy@theaiwrangler.comThis LoRA is a derivative of LTX-2 and is additionally subject to the
LTX-2 Community License Agreement. Users are independently responsible for complying with all applicable Lightricks licensing requirements.
