Convert Qwen-Rapid-AIO diffusion models from FP8 to NVFP4 format for faster inference on RTX 5090 (Blackwell) with custom calibration to preserve image quality.
Source Model: Qwen-Rapid-AIO-NSFW-v14.1 (27GB, FP8, LoRA-fused)
Inference Stack: ComfyUI on RTX 5090 with native NVFP4 support
Recommended Model: V3_NVFP4_custom_p999 ... best cost-performance balance
Why This Exists
The official comfy-dit-quantizer assumes BF16 source models. Our source is already FP8, which means:
FP8 weights must be upcast to FP32 before re-quantizing to NVFP4 (double quantization)
Text encoder FP8 layers are missing weight_scale tensors
Key names need normalization from model.diffusion_model.* to diffusers format
Base calibration data doesn't match our LoRA-merged model's activation distributions
This pipeline solves all four problems.
Recommended Model: V3_NVFP4_custom_p999
After benchmarking all versions across 100 prompts, V3 delivers the best cost-performance ratio: 1.84x speedup over FP8 baseline with the tightest calibration scales.
Model
Avg Time/Image
Speedup
NVFP4 Layers
FP8 Layers
Key Trait
FP8 Baseline
8.38s
1.00x
0
All
Reference quality
V2 (custom calib)
4.52s
1.85x
360
599
First custom calibration
V3 (p99.9 calib)
4.55s
1.84x
360
599
Best cost-performance
V4 (selective)
6.05s
1.39x
240
719
Better quality, slower
V5 (text quality)
8.39s
1.00x
180
779
Best quality, no speedup
V3 uses P99.9 percentile calibration from 53 sample generations with skin-focused prompts. The p99.9 method produces ~74% tighter scales than max aggregation, giving NVFP4 layers better dynamic range utilization without being dominated by rare outlier activations.
V4 and V5 trade speed for quality by keeping more layers at FP8. V5 in particular loses all speedup benefit, making it impractical for production use.
The Problem: Quality Degradation in NVFP4
Converting FP8 to NVFP4 introduces two compounding quality issues:
1. Double Quantization
The pipeline stacks two aggressive quantizations, losing ~5+ bits of precision from the original BF16:
Original BF16 (7 mantissa bits)
-> FP8 E4M3 (3 mantissa bits) ~4 bits lost
-> Upcast to FP32 No recovery, same 3 bits of real info
-> NVFP4 (2 bits, 4 quant levels) ~5+ bits total lost from original
The upcast to FP32 does NOT recover original precision. It just represents already-degraded FP8 values in a larger format.
Constraint: We cannot access the original BF16 weights or the original FP8 quantization scales. The Qwen-Rapid-AIO models are only distributed in FP8 format.
2. Calibration Mismatch
The base calibration (calibs/qwen-image-edit-2511.json) was generated from the original qwen-image-edit-2511 model, not our LoRA-merged variant. When LoRAs are merged (W' = W + BA), weight magnitudes and activation ranges shift significantly. Our custom calibration values were ~20x larger than the base calibration, confirming the mismatch.
Mismatch Type
Effect on Output
Scale too large
Activations quantized toward zero (underflow)
Scale too small
Activations clipped at max (overflow)
General mismatch
Poor utilization of NVFP4's 4 quantization levels
Improvement Journey
We iterated through 5 model versions, each addressing a specific quality or performance issue:
V1: Baseline NVFP4 (External Calibration)
Used calibration data from others, not matched to this LoRA-fused model
Noticeable quality degradation vs FP8 original
V2: Custom Calibration
Generated custom calibration with 10 sample generations
Custom values ~20x larger than base calibration, confirming LoRA merge changed activation distributions
Some improvement, but skin quality still worse than FP8
Three solutions remain available for teams needing even better quality:
Solution
Quality Impact
Effort
Performance Cost
Preserve FP8-to-FP8 layers
Good
Low
None
Custom calibration on actual model
High
Medium
None
Selective NVFP4 scope reduction
Good
Medium
Some slowdown
Preserve FP8-to-FP8 layers: The current pipeline re-quantizes FP8 layers through FP32 even when the target is FP8. Skipping this for FP8-targeted layers eliminates ~40% of unnecessary weight degradation with zero downside.
Full inference calibration: Running 256-512 diverse generations through the actual LoRA-merged model (varied prompts, image sizes, guidance scales, denoising steps) would produce the most accurate activation scales. The hook-based approach instruments the model during inference to capture per-layer amax values.
Selective NVFP4: Profiling which specific layers contribute most to quality degradation allows fine-tuning the NVFP4/FP8 boundary for the best quality-speed tradeoff on your specific use case.
