Wan2.2-T2V-A14B — QK-INT8 × PV-FP8 attention ablation (B200, 2026-07-21)
End-to-end video quality of low-bit attention kernels, measured against an
FA4-CuTe bf16 ground truth. 32 MovieGenVideoBench prompts, seed 42, 1280×720,
81 frames, UniPC 40 steps, shift 12, CFG 4.
Shared quantization recipe (all configs): QK INT8 — Q per-token scale, K
per-128-token scale; PV FP8 — V per-channel scale. Softmax row-sum computed on
tensor cores (l-MMA); exact exp = FA4-style EX2 + polynomial mix; split-P = 64.
Results (mean over 32 videos vs FA4 bf16 GT)
| Config | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|
| Sage2 baseline — no QK transform, K per-channel smooth, V per-channel smooth | 20.43 | 0.7402 | 0.2008 |
| Ours (exact exp) — + QK Hadamard, V balanced-kmeans reorder + per-128-block smooth | 21.24 | 0.7609 | 0.1757 |
| Ours + 50% linmant exp approx (hybrid50) | 19.97 | 0.7204 | 0.2136 |
| Ours + 100% linmant exp approx | 19.59 | 0.7049 | 0.2250 |
Paired per-video statistics (n = 32):
- Ours (exact) vs Sage2: +0.803 dB PSNR (SEM 0.444, wins 20/32, ≈1.8σ)
- linmant vs exact: −1.647 dB (SEM 0.463); hybrid50 vs exact: −1.264 dB (SEM 0.428)
Note: the 40-step diffusion chaos floor gives a per-video σ of ~2.3–2.9 dB, so
per-prompt deltas swing widely in both directions even when the mean improves —
the three sample prompts below were chosen to show exactly that spread.
Sample videos (videos/)
Naming: <prompt>__<config>.mp4, configs = gt_fa4 (reference), sage2,
ours_exact, ours_hybrid50, ours_linmant. PSNR vs GT per video:
| Prompt | sage2 | ours_exact | ours_hybrid50 | ours_linmant | novsmooth_linmant |
|---|
A_documentary-style_00250 (best case, ours +6.1 dB) | 20.81 | 26.91 | 24.59 | 24.89 | 23.76 |
A_macro_shot_in_real_00692 (median, ours +0.4 dB) | 16.38 | 16.82 | 18.59 | 18.56 | 15.32 |
A_steampunk-inspired_00733 (worst case, ours −5.1 dB) | 24.51 | 19.39 | 18.75 | 18.94 | — |
novsmooth_linmant is a stress probe: V-smooth fully OFF (per-channel FP8 V
only) + 100% linmant exp approx. No structural collapse — videos stay coherent;
the cost of removing V-smooth under linmant is content-dependent (−1.1 dB on
00250, −3.2 dB on 00692).
Hardware: NVIDIA B200 (sm100), CuTe-DSL kernels (tcgen05). Generated with the
fp8-attention max-approx kernel branch; balanced-kmeans V reorder reuses the
permutation across denoise steps inside an adaptive window.
