You can deploy this model and start earning money today!

lite-infer/
Qwen-Image-nunchaku-lite-nvfp4_r128-bnb4-text-encoder

Name: Qwen-Image-nunchaku-lite-nvfp4_r128-bnb4-text-encoder
Availability: InStock

Views

No views yet

Qwen-Image Nunchaku Lite FP4 R128 with BNB4 Text Encoder

[!IMPORTANT] Experimental: This Nunchaku Lite quantized Qwen-Image checkpoint is experimental. APIs and loading behavior may change.

This repository contains a full Diffusers pipeline for Qwen/Qwen-Image with:

a Nunchaku Lite quantized diffusion transformer
a prequantized bitsandbytes 4-bit NF4 text_encoder
inherited tokenizer, scheduler, and VAE from the Qwen-Image release

Quantization

The transformer uses a compact Nunchaku Lite quantization config stored in transformer/config.json.

Component	Method	Precision	Group size	Rank	Targets
Transformer `svdq_w4a4`	Nunchaku Lite	`fp4`	16	128	720
Transformer `awq_w4a16`	Nunchaku Lite	`int4`	64	-	120
Text encoder	bitsandbytes	`nf4` 4-bit	-	-	358 modules

Requirements

This checkpoint requires a Diffusers version that includes NunchakuLiteQuantizationConfig and the Nunchaku Lite loader path.

bash

pip install git+https://github.com/huggingface/diffusers
pip install kernels bitsandbytes

Use a CUDA-capable PyTorch environment. The quantized transformer uses kernels from the Hugging Face kernels package, and the text encoder uses bitsandbytes 4-bit modules.

Usage

python

import torch
from diffusers import QwenImagePipeline

pipe = QwenImagePipeline.from_pretrained(
    "lite-infer/Qwen-Image-nunchaku-lite-fp4_r128-bnb4-text-encoder",
    torch_dtype=torch.bfloat16,
).to("cuda")

image = pipe(
    prompt="A cinematic photo of a glass teapot on a wooden table, warm window light, detailed reflections, shallow depth of field",
    negative_prompt=" ",
    height=1024,
    width=1024,
    num_inference_steps=20,
    true_cfg_scale=4.0,
    generator=torch.Generator(device="cuda").manual_seed(42),
).images[0]

image.save("qwen-image-nunchaku-lite-fp4_r128-bnb4-text-encoder.png")

Evaluation

Evaluation was run with the following settings:

Hardware: NVIDIA GeForce RTX 5090 or equivalent Blackwell GPU with CUDA
Precision: torch_dtype=torch.bfloat16
Output: 1024x1024
Inference steps: 20
Guidance: true_cfg_scale=4.0
Negative prompt: " "
Seed: 42
Runs: 1 warmup run, 3 measured runs

The + torch.compile rows compile only pipe.transformer with mode=None and fullgraph=False.

[!NOTE] On Blackwell GPUs, the INT4 variants measured slower than FP4 in this benchmark. Prefer FP4 variants for latency-sensitive use on Blackwell-class hardware.

The Diffusers integration currently uses the Nunchaku Lite quantized modules and kernels, but does not include the original Nunchaku runtime's kernel/module fusing. The + torch.compile rows compile only the Diffusers transformer graph around those modules, so the speedup can be smaller than the original Nunchaku checkpoint benchmark below.

Variant	Generation latency	Peak allocated VRAM	Peak reserved VRAM
BF16 base	11.41s	57.94GB	58.80GB
INT4 R32 transformer + BNB4 text encoder	47.63s	21.22GB	22.55GB
INT4 R128 transformer + BNB4 text encoder	48.28s	22.45GB	23.06GB
FP4 R32 transformer + BNB4 text encoder	9.17s	21.58GB	22.11GB
FP4 R128 transformer + BNB4 text encoder	9.83s	22.76GB	23.56GB
BF16 base + torch.compile	10.06s	57.93GB	59.51GB
INT4 R32 transformer + BNB4 text encoder + torch.compile	46.41s	21.20GB	22.41GB
INT4 R128 transformer + BNB4 text encoder + torch.compile	47.06s	22.43GB	23.71GB
FP4 R32 transformer + BNB4 text encoder + torch.compile	8.14s	21.55GB	22.76GB
FP4 R128 transformer + BNB4 text encoder + torch.compile	8.83s	22.72GB	24.18GB

Original Nunchaku Checkpoint Benchmark

The original Nunchaku transformer was loaded with NunchakuQwenImageTransformer2DModel.from_pretrained(...) and inserted into the matching converted pipeline components so the text encoder, tokenizer, VAE, and scheduler matched the converted checkpoint.

Original checkpoint	Generation latency	Peak allocated VRAM	Peak reserved VRAM
`nunchaku-ai/nunchaku-qwen-image/svdq-int4_r32-qwen-image.safetensors`	36.47s	21.23GB	22.38GB
`nunchaku-ai/nunchaku-qwen-image/svdq-int4_r128-qwen-image.safetensors`	36.94s	22.26GB	22.82GB
`nunchaku-ai/nunchaku-qwen-image/svdq-fp4_r32-qwen-image.safetensors`	7.27s	21.56GB	22.10GB
`nunchaku-ai/nunchaku-qwen-image/svdq-fp4_r128-qwen-image.safetensors`	7.72s	22.64GB	23.28GB

Result Image

Conversion Comparison

Provenance

Base pipeline: Qwen/Qwen-Image
Source quantized checkpoint: nunchaku-ai/nunchaku-qwen-image, svdq-fp4_r128-qwen-image.safetensors

Limitations

This is an experimental quantized checkpoint.
The transformer and text encoder are quantized; the scheduler, tokenizer, and VAE are inherited from the base model.
Output quality and performance may differ from the original BF16 pipeline and from the original Nunchaku runtime.

License

This repository follows the Apache-2.0 license from the base Qwen-Image release.

lite-infer/Qwen-Image-nunchaku-lite-nvfp4_r128-bnb4-text-encoder