GGUF quantizations of
0xSero/gemma-4-21b-a4b-it-REAP, a 20% expert-pruned variant of
google/gemma-4-26b-a4b-it using the REAP (Router-weighted Expert Activation Pruning) method.
1# 1. Convert BF16 SafeTensors → GGUF
2python convert_hf_to_gguf.py 0xSero/gemma-4-21b-a4b-it-REAP \
3 --outfile gemma-4-21b-a4b-it-REAP-BF16.gguf \
4 --outtype bf16
5
6# 2. Quantize (example: Q4_K_M)
7llama-quantize gemma-4-21b-a4b-it-REAP-BF16.gguf \
8 gemma-4-21b-a4b-it-REAP-Q4_K_M.gguf Q4_K_M
1llama-cli \
2 -m gemma-4-21b-a4b-it-REAP-Q4_K_M.gguf \
3 -ngl 99 -c 4096 \
4 -p "Your prompt here"
1llama-server \
2 -m gemma-4-21b-a4b-it-REAP-Q4_K_M.gguf \
3 -ngl 99 -c 4096 \
4 --port 8080
0xSero/gemma-4-21b-a4b-it-REAP applies REAP expert pruning (
arXiv:2510.13999) to remove 20% of MoE experts (25 of 128 per layer) from Gemma 4 26B-A4B-it, while preserving routing behavior. Active parameters per token remain unchanged at ~4B. The result is an ~18% smaller model with near-identical generation quality across coding, math, and reasoning benchmarks.
