Custom Quantized Model
This repository contains a custom quantized version of the original model designed to reduce model size and VRAM use while preserving as much output quality as possible.
The model architecture, training, and capabilities come from the original model authors. This repository only changes how the model weights are stored and executed.
What was changed?
The original model was converted using post-training quantization. No additional model training or fine-tuning was performed.
Rather than applying the same precision to every layer, some releases may use mixed quantization. Different parts of the model are assigned different formats depending on their expected sensitivity to quantization.
For example:
- Sensitive layers may remain at higher precision.
- Attention layers may use FP8, MXFP8, W4A8, or another suitable format.
- Large MLP/FFN layers may use W4A8 or NVFP4.
- Layers that provide little size benefit or are known to be sensitive may remain at their source precision.
The goal is to get a better quality / VRAM / speed / model-size balance than simply quantizing every layer identically.
Quantization formats
Depending on the uploaded variant, the model may contain one or more of the following:
W4A8
4-bit weights with 8-bit activation quantization. This is generally used where better quality is preferred while still getting most of the storage benefit of 4-bit weights.
NVFP4
NVIDIA FP4 quantization intended primarily for Blackwell GPUs such as the RTX 5090. This can reduce runtime bandwidth and take advantage of Blackwell's native low-precision hardware.
MXFP8 / FP8
8-bit floating-point formats used for layers where additional precision is beneficial.
INT8
8-bit integer quantization for compatible layers and hardware.
Source precision
Some sensitive layers may remain BF16/FP16 or their original precision.
Mixed quantization
A mixed checkpoint may look conceptually like this:
1Large tolerant layers → NVFP4
2Quality-sensitive layers → W4A8
3Sensitive projections → MXFP8 / FP8
4Critical/small layers → Source precision
The exact layer assignment depends on the model and the specific preset used.
For Blackwell-targeted models, a Balanced configuration may combine NVFP4 and W4A8 instead of using one 4-bit format everywhere. NVFP4 is used where its speed and bandwidth advantages are useful, while W4A8 is retained where the additional activation precision may help preserve quality.
Preset philosophy
The quantization profiles are designed around different goals rather than simply using progressively lower precision everywhere.
Maximum Quality
Minimal aggressive quantization. Prioritizes fidelity.
Quality
Moderate size reduction while keeping most important layers at higher precision.
Balanced
Mixes high-quality 4-bit and 8-bit formats to provide a strong compromise between quality, size, VRAM use, and performance.
Fast
Favors formats that can take better advantage of the target GPU's low-precision hardware.
Small
Prioritizes model and VRAM size while retaining higher precision where it provides a meaningful quality benefit.
Maximum Compression
Uses aggressive low-precision quantization on most compatible layers.
Target hardware
Some variants are optimized for a particular GPU architecture.
For example:
1RTX 50 series / Blackwell
2 NVFP4
3 MXFP8
4 W4A8
5
6RTX 40 series / Ada
7 W4A8
8 FP8 E4M3FN
9 W4A4 ConvRot where appropriate
A checkpoint targeted for Blackwell may still load on other hardware depending on software support, but it may not provide the same performance advantages.
Quality
Quantization is lossy.
These models are intended to stay close to the original checkpoint, but small differences can occur in:
- fine detail
- textures
- color consistency
- motion
- prompt adherence
- character consistency
- difficult or long generations
Mixed quantization attempts to reduce these differences by keeping more sensitive parts of the model at higher precision.
Results can vary by prompt, sampler, resolution, number of frames, and workflow.
Compatibility
These checkpoints are intended for recent versions of ComfyUI with the required quantization support, including compatible versions of comfy-kitchen where applicable.
Some formats, particularly NVFP4 and MXFP8, depend on both software support and compatible NVIDIA hardware.
If the model fails to load, first make sure your ComfyUI and quantization dependencies are current.
Recommended use
Use the variant that best matches your hardware and priorities:
1Best fidelity → Maximum Quality / Quality
2General use → Balanced
3Highest throughput → Fast
4Low VRAM / storage → Small
5Minimum size → Maximum Compression
For an RTX 5090, the Blackwell-targeted Balanced or Fast variants are generally the most relevant starting points.
Source model
Original model:
[ORIGINAL MODEL NAME]
Original repository:
[ORIGINAL HUGGING FACE REPOSITORY]
Please review the original model card for:
- license
- intended uses
- limitations
- training information
- safety information
- attribution requirements
All credit for the original model belongs to its original authors.
This release
Variant: [MODEL / QUANT NAME]
Preset: [Balanced / Quality / Fast / etc.]
Target GPU: [RTX 5090 / Blackwell / Ada / Generic]
Approximate size: [XX GB]
Quantization: [e.g. Mixed NVFP4 + W4A8 + MXFP8]
Source precision: [BF16 / FP16 / other]
This is an unofficial community quantization unless otherwise stated.
