a Nunchaku Lite quantized diffusion transformer
a prequantized bitsandbytes 4-bit NF4 text_encoder
inherited tokenizer, prompt enhancer, scheduler, and VAE from the ERNIE-Image-Turbo release

The checkpoint safetensors were rewritten without safetensors metadata before upload.

Quantization

The transformer uses a compact Nunchaku Lite quantization config stored in transformer/config.json.

Component	Method	Precision	Group size	Rank	Targets
Transformer `svdq_w4a4`	Nunchaku Lite	`int4`	64	32	252
Transformer `awq_w4a16`	Nunchaku Lite	`int4`	64	-	6
Text encoder	bitsandbytes	`nf4` 4-bit	-	-	353 modules

Requirements

This checkpoint requires a Diffusers version that includes NunchakuLiteQuantizationConfig and the Nunchaku Lite loader path.

bash

pip install git+https://github.com/huggingface/diffusers
pip install kernels bitsandbytes

Use a CUDA-capable PyTorch environment. The quantized transformer uses kernels from the Hugging Face kernels package, and the text encoder uses bitsandbytes 4-bit modules.

Usage

python

import torch
from diffusers import ErnieImagePipeline

pipe = ErnieImagePipeline.from_pretrained(
    "rootonchair/ERNIE-Image-Turbo-nunchaku-lite-int4-bnb4-text-encoder",
    torch_dtype=torch.bfloat16,
).to("cuda")

image = pipe(
    prompt="A modern red armchair in a quiet studio, soft window light, realistic product photography",
    height=1024,
    width=1024,
    num_inference_steps=8,
    guidance_scale=1.0,
    use_pe=False,
).images[0]

image.save("ernie-image-turbo-nunchaku-lite-int4-bnb4-text-encoder.png")

Recommended ERNIE-Image-Turbo settings:

num_inference_steps=8
guidance_scale=1.0
use_pe=False for direct prompt usage, or use_pe=True to enable the inherited prompt enhancer

Evaluation

Evaluation was run on an NVIDIA RTX PRO 6000 Blackwell Server Edition with CUDA, torch_dtype=torch.bfloat16, 1024x1024 output, 8 inference steps, guidance_scale=1.0, and use_pe=False.

Variant	Load time	Generation latency	Avg eval latency	Peak allocated VRAM	Text encoder bnb modules
INT4 transformer + BNB4 text encoder	2.744s	11.323s	10.753s	9.330GB	353

Result Image

Prompt Edit Comparisons

Provenance

Base pipeline: baidu/ERNIE-Image-Turbo
Source quantized checkpoint: rootonchair/ERNIE-Image-Turbo-nunchaku-lite-int4

Limitations

This is an experimental quantized checkpoint.
The transformer and text encoder are quantized; the prompt enhancer, scheduler, tokenizer, and VAE are inherited from the base model.
save_pretrained() serialization for Nunchaku Lite models is not supported yet.
Output quality and performance may differ from the original BF16 transformer and text encoder.

License

This repository follows the Apache-2.0 license from the base ERNIE-Image-Turbo release.

lite-infer/ERNIE-Image-Turbo-nunchaku-lite-int4-bnb4-text-encoder

ERNIE-Image-Turbo Nunchaku Lite INT4 with BNB4 Text Encoder