Quantized weights repository: tonera/FLUX.2-klein-9B-kv-Nunchaku
Official full-precision upstream model: black-forest-labs/FLUX.2-klein-9b-kv
Quantized Transformer in this repo: svdq-<precision>_r32-FLUX.2-klein-9B-kv-Nunchaku.safetensors; fill <precision> with nunchaku.utils.get_precision() based on your environment and Nunchaku build, commonly fp4 or int4
Diffusers-side companion files (VAE, text encoder, etc.): kept consistent with the Hugging Face repo root, so you can load the pipeline from the same from_pretrained path

FLUX.2 [klein] 9B-KV is a KV-cache optimized variant of FLUX.2 [klein] 9B, mainly targeting multi-reference image editing and interactive workflows that reuse the same reference image repeatedly. It caches the key-value pairs of reference-image tokens in the first denoising step and reuses them in later steps, avoiding redundant computation. According to the upstream model card, this can provide up to 2.5x speedup in multi-reference editing scenarios.

Key Points From The Upstream Model Card

Capability scope: preserves the core capabilities of FLUX.2 klein 9B, including text-to-image, image-to-image, and multi-reference editing
KV-cache mechanism:
- Step 0: run the full forward pass, process reference-image tokens, and extract the KV cache
- Steps 1-3: reuse the cached KV pairs and skip repeated reference-image computation
Best-fit scenarios: multi-reference editing, generating multiple variants from the same reference image, and low-latency interactive image editing applications
Model shape: 9B flow model with an 8B Qwen3 text embedder, distilled for the common 4-step inference setup
Official inference stack: uses Flux2KleinKVPipeline in Diffusers
API availability: the upstream model card states that the model is available through the BFL API

Compared with the non-KV variant, the main difference is not a different task type, but reduced redundant computation when the same reference image participates in multiple denoising steps. If your workflow repeatedly reuses the same reference image, the KV variant is usually the better fit.

Quantization Quality (Excerpt From Internal Evaluation)

Metric	Mean	Median p50	p90
PSNR	22.10	22.35	25.58
SSIM	0.876	0.877	0.933
LPIPS	0.0734	0.0714	0.114

Performance benchmarks (RTX 5090 32GB, 8 steps, guidance scale = 1.0)

The benchmark data below is shared with FLUX.2-klein-9B-Nunchaku and can be used as a common performance reference for both models.

Base = black-forest-labs/FLUX.2-klein-9B
TE = svdq-int4-Qwen3-text-Nunchaku.safetensors
TR = svdq-{precision}_r32-FLUX.2-klein-9B-Nunchaku.safetensors
MCO = enable-model-cpu-offload
SCO = enable-sequential-cpu-offload

Image editing (1024x1024)

Setup	Mode	Peak VRAM	Throughput	Time to image	Throughput change vs. base	VRAM change vs. base
`Base`	`CUDA`	OOM	-	-	Baseline unavailable	Baseline unavailable
`Base`	`MCO`	20.18 GB	0.62 it/s	12 s	Baseline	Baseline
`Base`	`SCO`	2.55 GB	0.48 it/s	16 s	Baseline	Baseline
`Base + TE`	`CUDA`	25.60 GB	1.28 it/s	6 s	N/A (base OOM)	N/A
`Base + TE`	`MCO`	20.16 GB	0.83 it/s	9 s	+33.9%	-0.1%
`Base + TE`	`SCO`	6.08 GB	0.48 it/s	16 s	-0.5%	+138.4%
`Base + TR`	`CUDA`	24.51 GB	3.79 it/s	2 s	N/A (base OOM)	N/A
`Base + TR`	`MCO`	17.39 GB	1.08 it/s	7 s	+75.0%	-13.8%
`Base + TR`	`SCO`	4.35 GB	2.70 it/s	2 s	+461.6%	+70.6%
`Base + TR + TE`	`CUDA`	14.00 GB	3.81 it/s	2 s	N/A (base OOM)	N/A
`Base + TR + TE`	`MCO`	7.52 GB	1.88 it/s	4 s	+204.6%	-62.7%
`Base + TR + TE`	`SCO`	7.69 GB	2.68 it/s	2 s	+457.4%	+201.6%

Text-to-image (1024x1024)

Setup	Mode	Peak VRAM	Throughput	Time to image	Throughput change vs. base	VRAM change vs. base
`Base`	`CUDA`	OOM	-	-	Baseline unavailable	Baseline unavailable
`Base`	`MCO`	18.53 GB	0.83 it/s	9 s	Baseline	Baseline
`Base`	`SCO`	2.55 GB	0.62 it/s	12 s	Baseline	Baseline
`Base + TR + TE`	`CUDA`	15.21 GB	8.91 it/s	<1 s	N/A (base OOM)	N/A
`Base + TR + TE`	`MCO`	6.42 GB	2.60 it/s	3 s	+214.5%	-65.4%
`Base + TR + TE`	`SCO`	7.72 GB	3.00 it/s	2 s	+383.0%	+202.7%

Under MCO, Base + TR + TE remains the best-balanced option for speed and VRAM: for image editing, peak VRAM drops from 20.18 GB to 7.52 GB while throughput increases to about 3.05x the base model; for text-to-image, VRAM drops from 18.53 GB to 6.42 GB while throughput rises to about 3.15x. Under SCO, quantization still improves speed substantially, but peak VRAM does not necessarily keep decreasing because the base SCO path already compresses memory usage very aggressively.

Option 1 (for users comfortable with coding)

Engine: vitoom-nunchaku — community-maintained Nunchaku build with FLUX.2 Klein and Qwen3 text encoder support
Framework: a Diffusers build that supports Flux2KleinKVPipeline is required; the upstream example installs it from source:

pip install "git+https://github.com/huggingface/diffusers.git"

Install vitoom-nunchaku (recommended)

Upstream Nunchaku has not merged FLUX.2 Klein support for a long time (PR #926 still pending). Do not copy patch files manually. Install the prebuilt wheel from tonera/vitoom-nunchaku that matches your platform, Python, and CUDA.

Example (x86_64, Python 3.11, CUDA 13.0):

bash

pip install torch==2.11.* torchvision==0.26.* torchaudio==2.11.* \
  --index-url https://download.pytorch.org/whl/cu130

hf download tonera/vitoom-nunchaku \
  nunchaku-1.3.0.dev20260622+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl \
  --local-dir ./wheels

pip install ./wheels/nunchaku-1.3.0.dev20260622+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl

For cu128, cp310, or ARM64 aarch64 wheels, see the vitoom-nunchaku README.

Verify:

python -c "import nunchaku; from nunchaku import NunchakuFlux2Transformer2DModel; print(nunchaku.__version__)"

Minimal example (KV Pipeline + quantized Transformer)

The example below assumes vitoom-nunchaku is installed and weights are available locally or via tonera/FLUX.2-klein-9B-kv-Nunchaku. Replace REPO with your local path or Hugging Face model ID.

python

import torch
from diffusers import Flux2KleinKVPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuFlux2Transformer2DModel
from nunchaku.utils import get_precision

REPO = "tonera/FLUX.2-klein-9B-kv-Nunchaku"  # or a local absolute path
NAME = "FLUX.2-klein-9B-kv-Nunchaku"

transformer = NunchakuFlux2Transformer2DModel.from_pretrained(
    f"{REPO}/svdq-{get_precision()}_r32-{NAME}.safetensors",
    torch_dtype=torch.bfloat16,
)
pipe = Flux2KleinKVPipeline.from_pretrained(
    REPO, torch_dtype=torch.bfloat16, transformer=transformer
)

pipe.to("cuda")
# transformer.set_offload(
#     True, use_pin_memory=False, num_blocks_on_gpu=1
# )
# pipeline._exclude_from_cpu_offload.append("transformer")
# pipeline.enable_sequential_cpu_offload()

# Text-to-image (without passing image)
image = pipe(
    prompt="A cat holding a sign that says hello world",
    height=1024,
    width=1024,
    num_inference_steps=4,
    generator=torch.Generator(device="cuda").manual_seed(0),
).images[0]
image.save("t2i_output.png")

# Image editing / reference-image use case where KV cache is most relevant
ref = load_image("https://example.com/your_ref.png").convert("RGB")
image_kv = pipe(
    prompt="A cat dressed like a wizard",
    image=ref,
    height=1024,
    width=1024,
    num_inference_steps=4,
    generator=torch.Generator(device="cuda").manual_seed(0),
).images[0]
image_kv.save("kv_output.png")

If you generate multiple variants from the same reference image, the KV variant usually shows a clearer advantage. If memory is tight, you can also use strategies such as pipe.enable_model_cpu_offload().

If the steps above feel too difficult, install the vitoom platform instead (see below).

Option 2 (recommended: vitoom)

We recommend deploying via vitoom: it ships a complete vitoom-nunchaku runtime, Web UI, and multi-reference editing—no manual wheel install or file copying. See docker-usage-en.md.

bash

git clone https://github.com/tonera/vitoom.git
cd vitoom
python scripts/setup_vitoom.py
python scripts/load_vitoom_images.py --components backend,visual
docker compose up -d backend
docker compose -f docker-compose.inference.release.yml --profile visual up -d

In the Web UI: Models → download and activate tonera/FLUX.2-klein-9B-kv-Nunchaku → run in Image workspace.

Limitations, Hardware, And Compliance

Limitations: the upstream model card notes that the model is not intended to provide factual information; rendered text may be distorted; outputs may reflect biases in the training data; and prompt following is influenced by prompting style
Out-of-scope use: the model must not be used for unlawful, fraudulent, defamatory, harassing, or otherwise policy-violating scenarios
Hardware reference: the upstream model card lists about 29GB VRAM for the original KV version and targets RTX 5090 or above; after quantization, practical memory usage is about 17GB. If used together with tonera/Qwen3-text-Nunchaku, VRAM usage can be reduced substantially further
Responsible AI: Black Forest Labs states that it evaluated and mitigated risks including CSAM and NCII before release; see its safety materials for details. Safety issues can be reported to safety@blackforestlabs.ai

License And Notes

These quantized weights are derived from FLUX.2-klein-9b-kv. Usage must comply with the FLUX Non-Commercial License and Black Forest Labs acceptable-use policies. Please confirm licensing separately if you need commercial use.

The model card YAML uses license: other only because Hugging Face metadata fields are limited to a fixed enum and do not include a dedicated FLUX non-commercial option. The legally binding terms are the linked license above, not the placeholder value other itself.

tonera/FLUX.2-klein-9b-kv-Nunchaku

Model Overview (SVDQuant / Nunchaku / KV-Cache)

Model And Upstream