Views
No views yet
flux-2-klein-base-4b.safetensors, double_blocks.* key naming) —
not the diffusers-format transformer/diffusion_pytorch_model.safetensors in the same repo, which
uses different tensor names (x_embedder, context_embedder, etc.) and is not what ComfyUI's loader
expects.| File | Size | What it is | Derived from |
|---|---|---|---|
flux2-klein-base-4b_convrot_int8.safetensors | ~4.1 GB | Every quantization-eligible weight quantized to INT8 with ConvRot (group-wise Hadamard rotation, dynamic group size, 256 floor); modulation, embedding, and final-layer weights kept at source BF16 precision | flux-2-klein-base-4b.safetensors (native/BFL format, BF16) |
flux2-klein-base-4b_nvfp4_convrot_int8.safetensors | ~2.9 GB | Bulk weights (the middle double_blocks 1-3 and single_blocks 2-17) quantized to NVFP4; the first/last block of each stack (double_blocks 0,4 and single_blocks 0,1,18,19 — structurally the most precision-sensitive per common transformer-quantization heuristics) quantized to INT8 ConvRot instead of plain FP8; modulation, embedding, and final-layer weights kept at source BF16 precision | flux-2-klein-base-4b.safetensors (native/BFL format, BF16) |
flux2-klein-base-4b_mixed_int4_int8_convrot.safetensors | ~2.7 GB | Same layer split as the NVFP4 variant, but the 56 bulk layers (formerly NVFP4) are INT4 W4A4 ConvRot instead; the 24 edge-block layers stay INT8 ConvRot | flux-2-klein-base-4b.safetensors (native/BFL format, BF16) — the 56 bulk layers were quantized standalone from this BF16 source (not from the NVFP4 file) and spliced into a copy of flux2-klein-base-4b_nvfp4_convrot_int8.safetensors, replacing its NVFP4 tensors, to avoid compounding quantization error |
flux2-klein-base-4b_convrot_int8.safetensors: broadest compatibility — any modern GPU with usable
INT8 tensor-core throughput, no Blackwell requirement.flux2-klein-base-4b_nvfp4_convrot_int8.safetensors: requires a Blackwell GPU (SM ≥ 10.0/12.0) for
NVFP4 inference support.flux2-klein-base-4b_mixed_int4_int8_convrot.safetensors: no Blackwell dependency, but INT4
tensor-core throughput varies significantly by GPU generation — verify actual runtime behavior on
target hardware rather than assuming uniform benefit. Quality has not been evaluated (see
Verification below); treat this variant as experimental.convert_to_quant (ctq).1ctq -i flux-2-klein-base-4b.safetensors -o flux2-klein-base-4b_convrot_int8.safetensors \
2 --int8 --scaling-mode row --dynamic-convrot --convrot-group-size 256 \
3 --flux2 --comfy_quant --save-quant-metadata--flux2 exclusion preset kept the following layers at source BF16 precision (not quantized):
double_stream_modulation_img.lin.weight, double_stream_modulation_txt.lin.weight,
final_layer.adaLN_modulation.1.weight, final_layer.linear.weight, img_in.weight,
single_stream_modulation.lin.weight, time_in.in_layer.weight, time_in.out_layer.weight,
txt_in.weight — modulation, the final output layer, the image input embedder, time embedding, and
the text input embedder. 80 of 149 tensors were quantized; the remainder are these 9 excluded weights
plus non-weight tensors (norms, scales) that aren't quantization targets. Uses learned-rounding
optimization (SVD/AdaRound) — not a --simple/RTN build.1ctq -i flux-2-klein-base-4b.safetensors -o flux2-klein-base-4b_nvfp4_convrot_int8.safetensors \
2 --nvfp4 \
3 --custom-layers 'double_blocks\.(0|4)\..*\.weight$|single_blocks\.(0|1|18|19)\.linear[12]\.weight$' \
4 --custom-type int8 --custom-scaling-mode row --custom-convrot --custom-convrot-group-size 256 \
5 --flux2 --comfy_quant --save-quant-metadatadouble_blocks and single_blocks in this architecture have unequal role-separation: double_blocks
keep attention (*_attn.qkv/*_attn.proj) and MLP (*_mlp.0/*_mlp.2) as separate tensors, but
single_blocks.*.linear1/linear2 fuse attention and MLP into single tensors (standard FLUX
single-stream design), so a clean attention-vs-MLP split isn't possible there. This build instead
routes by block position: the first/last block of each stack (structurally the most
precision-sensitive per common transformer heuristics) stays INT8 ConvRot; the middle blocks go NVFP4.
56 of 80 quantization-eligible tensors went to NVFP4, 24 to INT8 ConvRot; the same 9 layers as the
ConvRot INT8 build stayed BF16. Uses learned-rounding optimization (SVD/AdaRound) — not a
--simple/RTN build.convert_to_quant checkout used for this build (--layer-config silently maps
"format": "nvfp4" to FP8 instead) meant this file was built with --custom-layers/--custom-type
rather than --layer-config; a fix is proposed in
silveroxides/convert_to_quant#56.1# 1. Extract just the 56 target tensors from the original BF16 source (not the NVFP4 file --
2# quantizing from BF16 avoids compounding error).
3python extract_nvfp4_candidates.py \
4 --src flux-2-klein-base-4b.safetensors \
5 --out flux2-klein-base-4b_nvfp4_candidates_bf16.safetensors
6
7# 2. Quantize the extracted subset directly to INT4 W4A4 ConvRot.
8ctq -i flux2-klein-base-4b_nvfp4_candidates_bf16.safetensors \
9 -o flux2-klein-base-4b_nvfp4_candidates_int4.safetensors \
10 --int4 --dynamic-convrot --convrot-group-size 256 \
11 --comfy_quant --save-quant-metadata
12
13# 3. Splice the INT4 tensors into a copy of flux2-klein-base-4b_nvfp4_convrot_int8.safetensors,
14# replacing the NVFP4 tensors at those keys. Updates both the per-tensor .comfy_quant blob
15# and the global __metadata__["_quantization_metadata"]["layers"] entry for every replaced
16# key -- ComfyUI's loader treats the global blob as authoritative for format detection.
17python splice_int4_into_nvfp4_convrot_int8.py \
18 --base flux2-klein-base-4b_nvfp4_convrot_int8.safetensors \
19 --int4 flux2-klein-base-4b_nvfp4_candidates_int4.safetensors \
20 --out flux2-klein-base-4b_mixed_int4_int8_convrot.safetensors_quantization_metadata (not just a successful command exit). Output quality has
not been separately assessed against the BF16 source for any variant — only functional correctness
(loads, runs, produces a coherent image matching the prompt). Visually, the mixed INT4/INT8 output
showed a noticeable loss of fine detail (e.g. mountain texture) compared to the INT8 and NVFP4
variants in a spot-check generation — consistent with the standing caveat that INT4 ConvRot quality
depends heavily on layer-selection and rounding method and hasn't been formally evaluated.LICENSE.md in this repo. This quantized derivative
is also distributed under Apache 2.0, per the terms of the source license. No files from the original
NOTICE were required (the source repo does not include a NOTICE file). This file has been modified
from the original via the quantization method described above.