GGUF quantizations of
0xSero/Qwen3.6-28B-REAP20-A3B, a 20% expert-pruned variant of
Qwen/Qwen3.6-35B-A3B using the REAP (Router-weighted Expert Activation Pruning) method.
1# 1. Download BF16 GGUF from source
2huggingface-cli download 0xSero/Qwen3.6-28B-REAP20-A3B-GGUF \
3 --include "model.bf16.gguf" --local-dir ./
4
5# 2. Download imatrix (for IQ quants)
6huggingface-cli download 0xSero/Qwen3.6-28B-REAP20-A3B-GGUF \
7 --include "imatrix.dat" --local-dir ./
8
9# 3. Quantize (example: Q4_K_M)
10llama-quantize model.bf16.gguf Qwen3.6-28B-REAP20-A3B-Q4_K_M.gguf Q4_K_M
11
12# 4. Quantize with imatrix (example: IQ3_XXS)
13llama-quantize --imatrix imatrix.dat model.bf16.gguf \
14 Qwen3.6-28B-REAP20-A3B-IQ3_XXS.gguf IQ3_XXS
1llama-cli \
2 -m Qwen3.6-28B-REAP20-A3B-Q4_K_M.gguf \
3 -ngl 99 -c 4096 \
4 -p "Your prompt here"
1llama-server \
2 -m Qwen3.6-28B-REAP20-A3B-Q4_K_M.gguf \
3 -ngl 99 -c 4096 \
4 --port 8080
0xSero/Qwen3.6-28B-REAP20-A3B applies REAP expert pruning (
arXiv:2510.13999) to remove 20% of MoE experts (51 of 256 per layer) from Qwen3.6-35B-A3B, while preserving routing behavior via router weight renormalization. Active parameters per token remain unchanged at ~3B. The result is a ~25% smaller model with competitive generation quality across coding, reasoning, and knowledge benchmarks.
