Gemma 4 21B-A4B-IT REAP — Q4_K_M GGUF
What is REAP?
Router-weighted Expert Activation Pruning — removes the 20% least important MoE experts (25 of 128 → 103 remaining) using calibration-based scoring. Unlike quantization, REAP removes entire experts while keeping the same active parameter count (~4B) per token.
Model Details
| Property | Value |
|---|
| Base Model | google/gemma-4-26b-a4b-it |
| Pruning | REAP 0.20 (103/128 experts) |
| Quantization | Q4_K_M (5.32 BPW) |
| File Size | 12.87 GB |
| Total Parameters | 20.77B |
| Active Parameters | ~4B per token |
| Context Window | 262,144 tokens |
| Architecture | Gemma4 MoE (30 layers) |
Performance
Tested on RTX 4070 Ti SUPER (16GB VRAM):
- Speed: 65-95 tokens/second
- VRAM Usage: ~14.8 GB (fits 16GB cards)
- Full GPU offload with (KV cache in system RAM)
How to Run (llama.cpp)
Conversion
Converted from BF16 safetensors → F16 GGUF using llama.cpp convert_hf_to_gguf.py, then quantized with llama-quantize to Q4_K_M.
