Unofficial community project. Not affiliated with, endorsed by, or produced
by Alibaba Group or the Wan-Video team. Built on their Apache-2.0
Wan2.1 release.
A 1.3 B student distilled from a Wan2.1-14B teacher that generates video
continuously rather than as a fixed clip: 750 ms of 640x368 video per block,
generated in 428 ms on one A100 80GB, extended block by block for as long as you
let it run. The text conditioning can be changed mid-stream without clearing the
K/V cache, so the scene continues instead of cutting.
Code, setup and documentation: JonathanColetti/LiveWan. This repository holds
weights and data only. It is not usable on its own.
Not audio
The model is text-to-video and was never trained to articulate speech. There is no
audio, no lip sync and no speech anywhere in this project. Text steers the scene.
A face that appears to talk is generating plausible mouth motion, not saying your
sentence.
Files
To run the model — 6.0 GB. This is what setup.sh fetches.
the four cached evaluation worlds — skip base-model generation entirely
samples/
16 MB
reference clips and analysis filmstrips from those worlds
To continue the run Not needed for inference. do not
download these unless you intend to train.
path
size
what
checkpoints/t14b_b64/latest.pt
10.6 GB
step 3000, carries its EMA copy. The comparison arm, and the weights a resume starts from
checkpoints/t14b_b64/opt_rank00{0..7}.pt
8 × 2.2 GB
FSDP-sharded AdamW state at step 3000, one shard per rank
Nothing here duplicates a public download. The Wan2.1 base model, its VAE and the
umt5-xxl encoder are not mirrored in this repository. setup.sh pulls
Wan-AI/Wan2.1-T2V-1.3B (17 GB)
from its own repo, and the 14B teacher is only needed if you are distilling.
The layout mirrors the code repository, so setup.sh downloads straight into place.
To take only part of it:
Needs ~10 GiB of VRAM. scripts/live_demo.py serves the same model as a steerable
browser stream.
--prompt-idx must match the world: world_p60.pt goes with --prompt-idx 60.
Indices run 0–95; anything above fails.
Training
3000 iterations of SF-DMD distillation from a Wan2.1-T2V-14B teacher into a
Wan2.1-T2V-1.3B student, effective batch 64 (8×H200, accum 8, FSDP), 41.6 hours
at 66.5 s/it, zero interventions. Losses do not decrease in this trainer and should
not (the critic is retrained every step, so the generator holds position against a
strengthening opponent).
Continuing the run
The optimizer shards make extending the run a resume rather than a restart.
Put latest.pt, all eight opt_rank*.pt and history.json in the same run
directory and:
ITERS=4000 ./run_b64_resume.sh
Four things decide whether this works:
Exactly 8 ranks. These are per-rank FSDP shards, not a consolidated state
dict. World size 8 is what wrote them and world size 8 is what can read them;
fewer ranks means larger shards, not smaller, so there is no repacking
shortcut. run_b64_resume.sh checks for all eight up front and refuses rather
than half-starting.
You resume from step 3000, not from the released step 2250. Optimizer state
and the EMA copy exist only for step 3000 — 2250 was archived _noema and its
EMA is gone. The released checkpoint is the better model; latest.pt is the
only resumable one.
--resume overwrites the checkpoint it resumed from. It reads
OUT/latest.pt and writes that same path at the first save.
run_b64_resume.sh preserves it as step003000.pt first — if you drive the
trainer directly instead, copy it aside yourself.
~30 GiB must be free transiently at each save, because the shards are
written as .tmp and renamed, so old and new coexist. Prune before the save,
not after; a post-save janitor cannot help, since the save is what runs out of
space.
The resume path is verified rather than assumed: killing a run at step 12 brought
AdamW's internal counters back as 16/18 rather than reset, and the real 750→3000
resume held loss_gen, loss_critic, dmd_grad_norm and gn_gen within one
standard deviation of the 60-point pre-crash reference. A subtly broken restore
shows up as a shifted mean or changed variance; neither appeared.
Verified
all 825 tensors load, zero non-finite values, weight norms 0.018–115.4
three correctness gates pass with their deliberately broken controls failing, both
before and after training
24 streaming cells across every arm and world met the real-time budget
same box, same seed → byte-identical output
Limits
Quality holds for roughly a minute. By 160 s sharpness has fallen to ~73% of
the world's.
Maximum 1024 latent frames (4.3 minutes at block 3), set by WanModel.freqs.
The pixel check has not been done. Checkpoint selection rests on proxy metrics,
and in this project nine automatic measurements have pointed the wrong way — every
one caught by looking at pixels at 1:1.
World 82 is unexplained: every trained checkpoint drifts on it while the
untrained baseline sits at −0.001.
The batch-64 premise is unsettled — confounded with having simply trained
longer.
Licence
Apache-2.0. Derived from Wan2.1
(Apache-2.0, Alibaba Group).
