A 4-bit-weight, 4-bit-activation quantization of Tongyi-MAI/Z-Image-Turbo via SVDQuant, in NVFP4 precision (E2M1 + per-16-K fp8_e4m3fn scales) with rank-128 SVD residual.

The DiT transformer is quantized; the Qwen3 text encoder and the VAE stay in BF16.

What's stored on disk

Canonical row-major NVFP4 layout. Not a nunchaku PTX-MMA-tile fragment — that is a kernel-specific permutation. Per-parameter:

param	shape	dtype
`qweight`	`[N, K/2]`	`int8` (FP4 nibble pack, low nibble = even-k)
`wscales`	`[K/16, N]`	`float8_e4m3fn`
`wcscales`	`[N]`	`bfloat16` (per-output-channel)
`wtscale`	`[1]`	`bfloat16` (global α)
`smooth_factor`	`[K]`	`bfloat16`
`proj_down`	`[K, R]`	`bfloat16` (SVD residual)
`proj_up`	`[N, R]`	`bfloat16` (SVD residual)

R = 128. The W4A4 reconstruction at runtime is

y = scaled_mma(quant(x · smooth), qweight, ascales, wscales) · wtscale · wcscales + (x @ proj_down) @ proj_upᵀ + bias

Per-component quantization_config in transformer/config.json restricts SVDQuant to the DiT only; the text encoder loads unquantized.

Why row-major, and what about consumer-card nunchaku kernels

nunchaku's W4A4 CUDA kernel consumes a PTX-MMA-tile-permuted fragment of qweight, wscales, proj_up, and proj_down. Shipping that permutation on disk would force every consumer to understand the nunchaku layout, even on hardware that does not use the nunchaku kernel.

This repository stores the canonical row-major form instead.

The native CuTe DSL kernel (datacenter Blackwell SM_100a / SM_103 in vLLM, vllm/model_executor/layers/quantization/kernels/svdquant/cute/) consumes the on-disk tensors directly with no repack.
The nunchaku kernel (consumer NVIDIA, SM_75 – SM_120) needs the PTX-MMA-tile layout; vLLM's SVDQuant linear method runs a one-shot pack at load time (vllm/model_executor/layers/quantization/svdquant.py::_pack_nvfp4_to_nunchaku_fragment). The pack/unpack pair is bit-exact and verified against nunchaku.ops.gemm.svdq_gemm_w4a4_cuda.

So the on-disk format does not leak backend choice. A future B200 + CuTe path and a current 4090 + nunchaku path read the same files.

Usage (vLLM + vllm-omni)

python

from vllm_omni.diffusion import OmniPipeline

pipe = OmniPipeline.from_pretrained(
    "ultranationalism/nunchaku-z-image-turbo-svdq",
    torch_dtype="bfloat16",
)
# Backend is picked automatically: native (B200/GB300) or nunchaku
# (Turing through consumer Blackwell). The selection is in
# `vllm/model_executor/layers/quantization/utils/svdquant_dispatch.py`.

image = pipe(
    "Young woman in a quiet rooftop garden at golden hour, photorealistic",
    height=1024, width=1024,
    num_inference_steps=9,   # 8 DiT forwards + 1 t=0
    guidance_scale=0.0,
).images[0]
image.save("out.png")

Hardware paths:

GPU	backend	notes
RTX 4090 / 5090 / 3090 (SM_8.x, SM_120)	`nunchaku`	requires `pip install nunchaku`; load-time repack to fragment
RTX 6000 Pro Blackwell (SM_120)	`nunchaku`	same as above
B200 / GB300 (SM_100a / SM_103)	`native`	CuTe DSL kernel, direct row-major consume
H100 / Hopper (SM_90)	unsupported	nunchaku does not target SM_90, native is SM_100+

Conversion source

Converter: vllm_omni/quantization/tools/convert_nunchaku_to_svdquant.py. Layout adapters live in vLLM at vllm/model_executor/layers/quantization/utils/svdquant_nvfp4_layout.py.

Input: nunchaku-tech/nunchaku-z-image-turbo. Each tensor is unpacked once from the nunchaku fragment to the row-major canonical form (the converter is the only consumer of the unpack_* direction; pack_* runs in vLLM at load time for the nunchaku backend).

proj_down carries a transpose quirk in nunchaku's pack_lowrank_weight(..., down=True) ([R, K] in / [K, R] out); the converter strips it so disk layout is plain [K, R]. Round-trip verified bit-exact across all stress shapes — see scratch/diag_proj_down_roundtrip.py in the working tree.

License

Apache-2.0, inherited from the base model.

About Z-Image-Turbo (upstream)

Everything below is reproduced from the upstream Z-Image-Turbo README so this card stands alone.

⚡️- Image
_{^{An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer}}

Official Site

GitHub

Z-Image is a 6B parameter image generation foundation model. Z-Image-Turbo is the distilled 8-NFE variant; details, model zoo, showcase, architecture, and benchmarks are at the official Hugging Face card linked above.

🔬 Decoupled-DMD

arXiv

The core few-step distillation algorithm behind the 8-step Z-Image-Turbo.

🤖 DMDR

arXiv

Reinforcement learning fused into post-training of the few-step model.

📜 Citation

bibtex

@article{team2025zimage,
  title={Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer},
  author={Z-Image Team},
  journal={arXiv preprint arXiv:2511.22699},
  year={2025}
}

@article{liu2025decoupled,
  title={Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield},
  author={Dongyang Liu and Peng Gao and David Liu and Ruoyi Du and Zhen Li and Qilong Wu and Xin Jin and Sihan Cao and Shifeng Zhang and Hongsheng Li and Steven Hoi},
  journal={arXiv preprint arXiv:2511.22677},
  year={2025}
}

@article{jiang2025distribution,
  title={Distribution Matching Distillation Meets Reinforcement Learning},
  author={Jiang, Dengyang and Liu, Dongyang and Wang, Zanyi and Wu, Qilong and Jin, Xin and Liu, David and Li, Zhen and Wang, Mengmeng and Gao, Peng and Yang, Harry},
  journal={arXiv preprint arXiv:2511.13649},
  year={2025}
}

@misc{li2024svdquant,
  title={SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models},
  author={Muyang Li and Yujun Lin and Zhekai Zhang and Tianle Cai and Xiuyu Li and Junxian Guo and Enze Xie and Chenlin Meng and Jun-Yan Zhu and Song Han},
  year={2024},
  eprint={2411.05007},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}

ultranationalism/nunchaku-z-image-turbo-svdq

Z-Image-Turbo · SVDQuant W4A4 (NVFP4)