MXFP4 quantization of MiniMax-M2.7 (228B params, 62 layers, 256 experts/layer, top-8 sigmoid routing).

All 15,872 MoE expert weights quantized to MXFP4. Attention, layer norms, embeddings, and router weights kept at original precision.

KLD is heavily right-skewed: median KLD is 0.031 nats/tok (5.6x lower than the mean). 96.6% of tokens have KLD < 1 nat. Only 69 out of 2048 eval tokens show significant divergence -- these are low-confidence positions where the model is already distributing probability across many candidates.

Error is diffuse across experts: per-expert quantization error analysis of all 15,872 experts shows extremely uniform error (std=0.000271, range 0.110--0.116). The 256-expert top-8 architecture is inherently quantization-tolerant -- each expert contributes ~1/8th of the output, so MXFP4 errors average out across the mixture.

MXFP4 block-32 quantization in compressed-tensors format:

Quantization is calibration-free (no calibration data needed). MXFP4 block-32 scaling is deterministic -- the shared exponent is derived directly from the max magnitude in each block.

Quantized with quant4.

Requires vLLM with MXFP4 compressed-tensors support and the CUTLASS FP4xFP8 kernel for Blackwell GPUs:

At 119 GB, this fits on 2x DGX Spark (2x 120 GB = 240 GB total) with ~100 GB remaining for KV cache, enabling long-context or multi-session serving that would be impossible with the 215 GB FP8 original.

Evaluated on WikiText-2 test set (2048 tokens) using layer-by-layer streaming inference with MiniMaxLayerRunner. Both models run identical forward passes; logits compared token-by-token.

Based on MiniMax-M2.7 by MiniMax. Original model license applies.