Qwen-Image-Edit-Rapid-AIO-NSFW-v23 — NVFP4 GGUF

Components

Hardware Requirements

• GPU: NVIDIA Blackwell (RTX 50-series) with CUDA 13.0+
• VRAM: ~10 GB for diffusion model + text encoder + VAE (NVFP4 saves ~4 GB vs FP8)
• System RAM: 32 GB+ recommended

Usage with ComfyUI

1. Install ComfyUI-GGUF
2. Place qwen-v23-diffusion-NVFP4.gguf in ComfyUI/models/checkpoints/
3. Place text_encoder/text_encoder-NVFP4.gguf in ComfyUI/models/clip/
4. Place vae/vae.safetensors in ComfyUI/models/vae/
5. Load the checkpoint normally — ComfyUI-GGUF will automatically detect the NVFP4 format

Quantization Details

Conversion Process

1. Split into three components (diffusion, text encoder, VAE)
2. Each component's FP8 tensors manually converted to F16 via streaming binary I/O with row-chunked FP8→F16 conversion to avoid OOM on large tensors
3. Written as F16 GGUF using gguf Python library (v0.19.0)
4. Quantized to NVFP4 using llama-quantize.exe (commit dd7cad7, CUDA 13.2)
5. Architecture binary-patched in output GGUF to match expected ComfyUI-GGUF target

Text Encoder Notes

• The text encoder (Qwen2.5-VL-7B, 7.75 GB F8_E4M3 → 3.84 GB NVFP4) uses the qwen2 GGUF architecture
• Input GGUF was written with both qwen2.* and llama.* metadata keys for quantizer compatibility, then patched to qwen2 after quantization
• Row-chunked FP8→F16 conversion (256 rows at a time) prevents OOM on the 545M-element embedding table
• Embedding weights are tied (lm_head shares weights with token_embd) — standard for Qwen2 models

Available Quantizations

Notes

• This is a NSFW checkpoint. Use at your own discretion.
• The text encoder is now also NVFP4 GGUF, saving ~4 GB (was 7.75 GB as safetensors) while remaining compatible with ComfyUI-GGUF.
• The VAE remains in safetensors format (BF16) for maximum compatibility.
• NVFP4 is a lossy quantization — expect minor quality degradation compared to the original FP8 model.
• Requires NVIDIA Blackwell GPU (RTX 50-series) with CUDA 13.0+ for NVFP4 compute support.

Links

• Original model: Phr00t/Qwen-Image-Edit-Rapid-AIO-NSFW-v23
• ComfyUI-GGUF: city96/ComfyUI-GGUF
• llama.cpp: ggml-org/llama.cpp