Safer memory-efficient compromise. It protects the last 16 decoder layers' o/down projections and keeps much lower storage than the full o/down guard.

This repository is part of the fixed SDNQ 4-bit HiDream O1 quantization set. The previous broad 4-bit recipes produced a visible tiled/grid artifact. The fix keeps the sensitive decoder projection path in higher precision, especially model.language_model.layers.*.mlp.down_proj.weight.

Comparison

Benchmarks were run on an NVIDIA RTX PRO 6000 Blackwell Workstation Edition with the HiDream O1 repository inference path, BF16 runtime, full-model 50-step default scheduler, guidance scale 5, shift 3, and flash attention disabled for parity. The requested 1024x1024 size is snapped by O1 to 2048x2048.

Model	Best for	Avg gen s	Gen time vs BF16	Peak alloc GiB	VRAM saved	Param storage GiB	Storage saved	Quantized layers	Quantized params B
Original BF16	Baseline quality/reference	28.91	-	17.38	-	16.40	-	-	0.00
Dynamic UINT4 threshold 1e-2, fixed	Fast balanced	30.62	+6%	10.88	+37%	9.87	+40%	int5:31, uint4:265	5.10
Static UINT4 + SVD r32, o/down BF16 guard	Safe default	32.30	+12%	10.74	+38%	9.71	+41%	uint4:296	5.10
Static UINT4 + SVD r32, down_proj BF16	Minimal fix	32.62	+13%	9.94	+43%	8.92	+46%	uint4:332	5.71
Static UINT4 + SVD r32, last 8 o/down BF16	Lowest VRAM	33.38	+15%	8.26	+52%	7.23	+56%	uint4:352	6.98
Static UINT4 + SVD r32, last 16 o/down BF16	Memory/quality	33.11	+15%	8.97	+48%	7.94	+52%	uint4:336	6.44

Variant Strengths

Dynamic UINT4 threshold 1e-2, fixed: Fastest balanced fixed variant. Dynamic quantization keeps the known artifact-producing down/output projections unquantized and raises difficult layers automatically.
Static UINT4 + SVD r32, o/down BF16 guard: Conservative default. Keeps both attention output and MLP down projections in BF16, the visually accepted fix for the tiled-grid artifact.
Static UINT4 + SVD r32, down_proj BF16: Smallest root-cause fix. Only MLP down projections are kept in BF16 beyond the standard output/embed skips; this isolates down_proj as the main grid culprit.
Static UINT4 + SVD r32, last 8 o/down BF16: Most memory-efficient clean-looking compromise from the matrix. It protects only the last 8 decoder layers' o/down projections.
Static UINT4 + SVD r32, last 16 o/down BF16: Safer memory-efficient compromise. It protects the last 16 decoder layers' o/down projections and keeps much lower storage than the full o/down guard.

This Variant

Source model: HiDream-ai/HiDream-O1-Image
Source snapshot: 407e54f7dc519bab53c9686f03489043065b0f81
Recipe: r32_last16_o_down_bf16
SDNQ layer counts: {"uint4": 336}
Quantized parameter counts: {"uint4": 6443528192}
Benchmark average generation time: 33.11s
Benchmark peak allocated VRAM: 8.97 GiB
Saved parameter storage: 7.94 GiB
10-demo average generation time: 33.75s
10-demo peak allocated VRAM: 9.21 GiB

Demo Comparisons

Each image in comparison/ is a pairwise original BF16 output next to this quantized variant with the same prompt, seed, and sampler settings.

Usage

bash

pip install sdnq torch transformers diffusers accelerate einops pillow scipy torchvision
git clone https://github.com/HiDream-ai/HiDream-O1-Image
cd HiDream-O1-Image

python

import torch
import sdnq
from transformers import AutoProcessor
from models.qwen3_vl_transformers import Qwen3VLForConditionalGeneration

model_id = "WaveCut/HiDream-O1-Image-SDNQ-uint4-svd-r32-last16-odown-bf16"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="cuda",
).eval()

Files

quantization_config.json - saved SDNQ config.
quantization_summary.json - quantized layer/parameter/storage summary.
benchmark_summary.json - matrix metrics plus 10-demo generation metrics.
comparison/00.jpg ... comparison/09.jpg - pairwise original vs quantized comparisons.
comparison/contact_sheet.jpg - compact overview of all 10 comparisons.