The transformer uses quant_method: nunchaku_lite, NVFP4 SVDQ with group size 16, runtime rank 64, 418 SVDQ targets, and 76 AWQ W4A16 targets. The CLIP encoder is copied from the base model and T5 text_encoder_2 is BitsAndBytes 4-bit NF4. Fused QKV modules are split in logical tensor layout; single-block proj_out is merged from attention and MLP projections; low-rank tensors are logically padded to rank 64. NVFP4 outer scales are reconciled without overflowing FP8 group scales.

Benchmark

Checkpoint	Latency	Max VRAM
Converted Diffusers Nunchaku Lite NVFP4 r32 + BNB4 T5	8.02 s (stdev 0.02 s)	16.75 GiB
Original Nunchaku NVFP4 r32 + BF16 T5	4.55 s (stdev 0.00 s)	21.01 GiB

RTX 5090, 1024×1024, 28 steps, guidance scale 3.5, one warmup and three measured runs, full GPU placement.

Output Comparison

Native reference (left) and converted output (right)

Both images use the same prompt, seed 0, scheduler, resolution, and step count. The native and converted NVFP4 images use identical inputs. Pixel MAE is 2.17 and RMSE is 4.93.

Run

Requires the Hugging Face kernels package and a Blackwell NVIDIA GPU for NVFP4 kernels.

python

import torch
from diffusers import FluxPipeline

pipe = FluxPipeline.from_pretrained(
    "lite-infer/flux.1-krea-dev-nunchaku-lite-nvfp4_r32-bnb4-text-encoder",
    torch_dtype=torch.bfloat16,
).to("cuda")

image = pipe(
    prompt='A cinematic photograph of a red fox standing in a misty forest at sunrise, detailed fur, volumetric light',
    generator=torch.Generator("cuda").manual_seed(0),
    width=1024,
    height=1024,
    num_inference_steps=28,
    guidance_scale=3.5,
).images[0]
image.save("output.png")