You can deploy this model and start earning money today!

leowu0511/
cuda_speed_ntu

Name: cuda_speed_ntu
Availability: InStock

Views

No views yet

Qwen-Image-Edit-Rapid-AIO-NSFW-v23 — NVFP4 GGUF

This repository contains NVFP4 (4-bit NVIDIA FP4) quantized GGUF components of the Phr00t/Qwen-Image-Edit-Rapid-AIO-NSFW-v23 merged checkpoint, converted for use with ComfyUI-GGUF.

File	Size	Description
`qwen-v23-diffusion-NVFP4.gguf`	11.5 GB	MMDiT diffusion backbone (~20B params) — NVFP4 quantized
`text_encoder/text_encoder-NVFP4.gguf`	3.84 GB	Qwen2.5-VL-7B text encoder — NVFP4 quantized
`vae/vae.safetensors`	0.25 GB	VAE decoder (BF16, original format)

GPU: NVIDIA Blackwell (RTX 50-series) with CUDA 13.0+
VRAM: ~10 GB for diffusion model + text encoder + VAE (NVFP4 saves ~4 GB vs FP8)
System RAM: 32 GB+ recommended

Install ComfyUI-GGUF
Place qwen-v23-diffusion-NVFP4.gguf in ComfyUI/models/checkpoints/
Place text_encoder/text_encoder-NVFP4.gguf in ComfyUI/models/clip/
Place vae/vae.safetensors in ComfyUI/models/vae/
Load the checkpoint normally — ComfyUI-GGUF will automatically detect the NVFP4 format

The original FP8 merged checkpoint was:

Split into three components (diffusion, text encoder, VAE)
Each component's FP8 tensors manually converted to F16 via streaming binary I/O with row-chunked FP8→F16 conversion to avoid OOM on large tensors
Written as F16 GGUF using gguf Python library (v0.19.0)
Quantized to NVFP4 using llama-quantize.exe (commit dd7cad7, CUDA 13.2)
Architecture binary-patched in output GGUF to match expected ComfyUI-GGUF target

The text encoder (Qwen2.5-VL-7B, 7.75 GB F8_E4M3 → 3.84 GB NVFP4) uses the qwen2 GGUF architecture
Input GGUF was written with both qwen2.* and llama.* metadata keys for quantizer compatibility, then patched to qwen2 after quantization
Row-chunked FP8→F16 conversion (256 rows at a time) prevents OOM on the 545M-element embedding table
Embedding weights are tied (lm_head shares weights with token_embd) — standard for Qwen2 models

Currently only NVFP4 is provided. Other quantizations (Q4_K_M, Q5_K_M, Q6_K, Q8_0) are possible using the same toolchain.

This is a NSFW checkpoint. Use at your own discretion.
The text encoder is now also NVFP4 GGUF, saving ~4 GB (was 7.75 GB as safetensors) while remaining compatible with ComfyUI-GGUF.
The VAE remains in safetensors format (BF16) for maximum compatibility.
NVFP4 is a lossy quantization — expect minor quality degradation compared to the original FP8 model.
Requires NVIDIA Blackwell GPU (RTX 50-series) with CUDA 13.0+ for NVFP4 compute support.