NVFP4 mixed-precision GGUF quantization of the REAP-40% pruned MiniMax M2.7 model, optimised for NVIDIA Blackwell GPUs. The NVFP4 tensor types use native Blackwell FP4 hardware acceleration for maximum inference speed, while smaller tensors (norms, biases, router, embeddings) are preserved in Q4_K for accuracy.

About the Source Model

m51Lab-MiniMax-M2.7-REAP-139B-A10B is a REAP-40% pruned variant of MiniMax M2.7, reducing total parameters from 229B to 139B while preserving the 10B active parameters per token. REAP (Router-weighted Expert Activation Pruning) prunes 40% of experts per MoE block (256 → 154) based on router-gated activation patterns.

Base model: MiniMaxAI/MiniMax-M2.7 (229B MoE, 62 layers)
Pruning method: REAP (Lasby et al., 2025, arXiv:2510.13999)
Pruning rate: 40% of experts per MoE block
Active parameters: ~10B per token
Architecture: Sparse MoE decoder-only Transformer
- Hidden dim: 3072
- Attention heads: 48 query / 8 KV (GQA)
- Experts: 154 per block (top-8 activated)
- Expert routing: Sigmoid gating with learnable bias
- Context length: 196,608 tokens
- Vocab: 200,064 (GPT-2 tokenizer)

Evaluation (from source)

HumanEval pass@1 (on completed): 83.3% (90/108)

All evaluations conducted on the Q4_K_M base variant. NVFP4 mixed quantization targets the same tensor patterns used by other Q4_K_M quality benchmarks, so quality should be equivalent or better (NVFP4 at 4.52 BPW vs Q4_K_M at 4.84 BPW on the output, but with hardware-accelerated inference on Blackwell).

Original Creator: m51Lab / dervig | License: Modified MIT (inherited from MiniMaxAI/MiniMax-M2.7)

Quantization Details

This is a mixed NVFP4/Q4_K quantization created using llama-quantize with --tensor-type-file on a Q4_K_M imatrix-quantized base.

Tensor Mapping

Tensor Pattern	Quantization	Rationale
`attn_q`	NVFP4	Large attention projection — benefits from FP4 hardware acceleration
`attn_k`	NVFP4	Attention key projection
`attn_v`	NVFP4	Attention value projection
`attn_output`	NVFP4	Attention output projection — large weight matrix
`ffn_gate_exps`	NVFP4	MoE gate expert weights — massive (1536 × 3072 × 154)
`ffn_up_exps`	NVFP4	MoE up expert weights — massive
`ffn_down_exps`	NVFP4	MoE down expert weights — massive (was Q6_K in base, significant size reduction)
All norms, biases, router, embeddings	Q4_K (fallthrough)	Small tensors, kept in Q4_K for accuracy

Size Comparison

Quantization	Size	BPW	Notes
Source (Q4_K_M imatrix)	79 GB	4.84	From mradermacher
This (NVFP4 mixed)	74 GB	4.52	~6.7% smaller, hardware-accelerated

The size reduction comes primarily from ffn_down_exps tensors which were Q6_K in the base (568 MB each) and are now NVFP4 (390 MB each) — a 31% reduction on those 62 tensors alone.

Performance

Benchmarked on single NVIDIA RTX PRO 6000 Blackwell Workstation Edition (97 GB VRAM) with llama.cpp build 9253 (commit 29f148222), CUDA sm_120, native Blackwell FP4 acceleration. Strict GPU 0 isolation (CUDA_VISIBLE_DEVICES=0).

Recommended Configuration

Setting	Value	Notes
Context size	80,000 tokens	Sweet spot; max f16 KV is 88K (88K barely fits, 90K+ OOMs)
Batch size	512	Decode is bandwidth-bound; b=512 == b=4096
KV cache type	f16 / f16	~12% faster than q8_0 on this MoE; f16 satisfies the no-below-q8 quality floor
Flash attention	on	Required for Blackwell
Speculative decoding	ngram-mod (n_match=24, n_min=4, n_max=48)	MTP heads stripped by REAP — n-gram is the only option. ~25% accept rate
Split mode	none (single GPU)	All on RTX 6000 Pro
Main GPU	0

Measured Throughput (canonical, "Give me a history of dotnet" prompt, 300 generated tokens)

Config	VRAM (MiB)	Prefill (tok/s)	Generation (tok/s)
20K ctx, f16 KV, ngram	80,577	443.0	132.9
20K ctx, f16 KV, no spec	80,973	437.1	133.6
20K ctx, q8_0 KV, no spec	78,287	404.7	120.0
80K ctx, f16 KV, ngram	95,091	—	132.7
80K ctx, q8_0 KV, no spec	86,389	404.3	120.2
131K ctx, q8_0 KV, no spec	92,967	404.1	120.1

Context Ceiling Analysis (f16 KV)

Context	Result	VRAM (MiB)	Notes
80,000	OK	95,091	Sweet spot — full perf, 2.8 GB headroom
85,000	OK	96,331	1.5 GB headroom; full perf
86,000	OK	96,511	Full perf
87,000	OK	96,759	Full perf, 1.1 GB headroom (recommended max)
88,000	OK	97,007	880 MiB headroom; prefill drops 23%, gen drops 14% under memory pressure
90,000	OOM	—	`cudaMalloc failed: out of memory` at load (allocating 396 MiB)
131,000 (q8_0 KV)	OK	92,967	Need q8_0 KV to fit; f16 KV needs ~31 GiB for KV cache alone

Key Findings

Decode is purely bandwidth-bound: batch size 512–4096 all yield ~120 tok/s with q8_0 KV and ~133 tok/s with f16 KV. Larger batch sizes give no throughput benefit.
f16 KV is faster than q8_0 on this MoE on Blackwell: ~11–13% decode speedup, with zero quality concern (f16 > q8_0 by definition, so it satisfies the no-below-q8 constraint).
Context scaling doesn't affect decode speed: 20K and 131K contexts both yield ~120 tok/s (q8_0 KV) — context only affects VRAM and prefill.
f16 KV costs exactly 2× q8_0 VRAM: 128 MiB/1K tokens for q8_0, 256 MiB/1K tokens for f16.
Bandwidth ceiling is ~135 tok/s on this single 6000 Pro — no spec, batch, or context tuning exceeds this.

NVFP4 on Blackwell uses native FP4 tensor core acceleration, providing significantly higher throughput than equivalent bit-width integer quants on the same hardware. The expert weights (ffn_*_exps) — which are the largest tensors in the model — benefit most from this acceleration.

Usage

llama.cpp Server (Recommended)

bash

llama-server \
  -m m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf \
  --host 127.0.0.1 --port 8080 \
  -ngl 99 \
  -c 80000 \
  -b 512 -ub 512 \
  --threads 64 \
  --flash-attn on \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --split-mode none \
  --main-gpu 0 \
  --cont-batching \
  --no-warmup \
  --spec-type ngram-mod \
  --spec-ngram-mod-n-match 24 \
  --spec-ngram-mod-n-min 4 \
  --spec-ngram-mod-n-max 48

For maximum context (use q8_0 KV to fit 131K):

bash

  -c 131000 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0
  # (omit the --spec-type args for max context + max speed; f16 KV hits OOM at 90K+)

Recommended Sampling Parameters

Temperature: 0.6–1.0
top_p: 0.95
top_k: 20–40
min_p: 0.01
repeat_penalty: 1.1

Speculative Decoding

This model does not include MTP (Multi-Token Prediction) heads — the REAP pruning process does not preserve the MTP module from the original MiniMax M2.7. N-gram speculative decoding is supported and yields ~25% accept rate, contributing ~3-5 tok/s extra:

bash

llama-server \
  -m m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf \
  -ngl 99 -c 20000 \
  --spec-type ngram-mod \
  --spec-ngram-mod-n-match 24 \
  --spec-ngram-mod-n-min 4 \
  --spec-ngram-mod-n-max 48

Hardware Requirements

Required: NVIDIA Blackwell GPU (RTX 5090, RTX PRO 6000, B200, etc.) for NVFP4 hardware acceleration
VRAM: ~62 GB minimum for model weights (single GPU), additional VRAM for KV cache
- q8_0 KV: 128 MiB per 1K context tokens
- f16 KV: 256 MiB per 1K context tokens
Tested: Single RTX PRO 6000 (97 GB) — full benchmark sweep at contexts 20K–131K
System RAM: 128 GB+ recommended
CPU: 32+ cores recommended (Threadripper-class) for prompt cache management

Reproduction

bash

# 1. Download the Q4_K_M imatrix source and imatrix file
hf download mradermacher/m51Lab-MiniMax-M2.7-REAP-139B-A10B-i1-GGUF \
  m51Lab-MiniMax-M2.7-REAP-139B-A10B.i1-Q4_K_M.gguf \
  m51Lab-MiniMax-M2.7-REAP-139B-A10B.imatrix.gguf \
  --local-dir ./source

# 2. Create tensor-type file
cat > nvfp4-tensor-types.txt << 'EOF'
attn_q=nvfp4
attn_k=nvfp4
attn_v=nvfp4
attn_output=nvfp4
ffn_gate_exps=nvfp4
ffn_up_exps=nvfp4
ffn_down_exps=nvfp4
EOF

# 3. Quantize
llama-quantize \
  --allow-requantize \
  --imatrix ./source/m51Lab-MiniMax-M2.7-REAP-139B-A10B.imatrix.gguf \
  --tensor-type-file nvfp4-tensor-types.txt \
  ./source/m51Lab-MiniMax-M2.7-REAP-139B-A10B.i1-Q4_K_M.gguf \
  m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf \
  Q4_K

File Information

Model file: m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf (~74 GB)
Format: GGUF v3
Architecture: minimax-m2
Quantization: Mixed NVFP4 (large weights) / Q4_K (small weights)
No mmproj required — text-only model

Acknowledgments

m51Lab — REAP-40% pruning of MiniMax M2.7 and base GGUF conversion
dervig — Publishing the pruned model on HuggingFace
MiniMax AI — base MiniMax M2.7 model
mradermacher — Q4_K_M imatrix quantization and i1-GGUF variant
ggml-org/llama.cpp — NVFP4 quantization support and inference engine (build 9253+)
Cerebras Research — REAP method (Lasby et al., 2025)

License

Modified MIT License — inherited from MiniMaxAI/MiniMax-M2.7. See the base model repository for full license terms.

Citation

bibtex

@article{lasby2025reap,
  title   = {REAP the Experts: Why Pruning Prevails for One-Shot MoE compression},
  author  = {Lasby, Mike and Hussein, Ahmed and Spyra, Jayden and Chkroun, Ivan
             and Suleiman, Oriol Sans and Ioannou, Nikoli and Hyder, Ammar Ali
             and Jacobs, Sam and Chaturvedi, Sachin and Mishra, Shreyanshu
             and Aboutalebi, Hossei and Rugol, Vasileios},
  journal = {arXiv preprint arXiv:2510.13999},
  year    = {2025}
}

@misc{minimax_m2_7,
  title  = {MiniMax-M2.7},
  author = {MiniMax AI},
  year   = {2026},
  url    = {https://huggingface.co/MiniMaxAI/MiniMax-M2.7}
}

s-batman/m51Lab-MiniMax-M2.7-REAP-139B-A10B-NVFP4-GGUF

m51Lab-MiniMax-M2.7-REAP-139B-A10B-NVFP4-GGUF

About the Source Model

Evaluation (from source)

Quantization Details

Tensor Mapping

Size Comparison

Performance

Recommended Configuration

Measured Throughput (canonical, "Give me a history of dotnet" prompt, 300 generated tokens)

Context Ceiling Analysis (f16 KV)

Key Findings

Usage

llama.cpp Server (Recommended)

Recommended Sampling Parameters

Speculative Decoding

Hardware Requirements

Reproduction

File Information

Acknowledgments

License

Citation

s-batman/
m51Lab-MiniMax-M2.7-REAP-139B-A10B-NVFP4-GGUF