4-bit weight quantization of MiniMax-M3 produced with GPTQ via llm-compressor, stored in the compressed-tensors pack-quantized format for serving with vLLM / SGLang. Routed-expert weights keep the base w1/w2/w3 naming, so the checkpoint loads against the native M3 architecture.

Quantization


Scheme	W4A16 (4-bit weights, 16-bit activations)
Weights	int4, symmetric, group size 128
Embeddings	int4, group size 64
`lm_head`	bf16 (kept full precision)
Left in bf16	router gates, lightning-indexer projections, RMSNorms, vision tower
KV cache	bf16
Method	GPTQ — Hessian error minimization with cross-layer error propagation
Format	compressed-tensors (`pack-quantized`)

All 60 decoder layers are quantized (3 dense + 57 sparse/MoE, 128 experts each).

Calibration

Calibrated on deployment-realistic agentic trajectories — 427 multi-turn tool-use rollouts (~8M tokens) rendered through the M3 chat template at up to 32k sequence length, padded with a small amount of general-instruction data. The goal is to match the model's real serving distribution (multi-turn, tool-calling) rather than generic web text.

Quality

Mean weight-reconstruction error vs the bf16 base is ≈0.125 relative (cosine ≈ 0.993) — the expected magnitude for int4 group-128 quantization. This is a weight-space measure; evaluate downstream quality on your own target benchmark.

Usage

bash

# requires a vLLM build with MiniMax-M3 support
vllm serve Sebesky/MiniMax-M3-W4A16-GPTQ

python

from transformers import AutoModelForImageTextToText, AutoTokenizer
model = AutoModelForImageTextToText.from_pretrained("Sebesky/MiniMax-M3-W4A16-GPTQ", device_map="auto")
tok = AutoTokenizer.from_pretrained("Sebesky/MiniMax-M3-W4A16-GPTQ")

License

Inherits the MiniMax Community License from the base model (non-commercial use).