Views
No views yet
| Component | Precision |
|---|---|
Routed experts — gate_proj / up_proj / down_proj (128 experts x 57 layers) | NVFP4 (block-16 E2M1, FP8-E4M3 block scales, FP32 global scale) |
Dense-MLP (layers 0-2) & shared-expert down_proj | NVFP4 |
Dense-MLP (layers 0-2) & shared-expert gate_up_proj | BF16 — see note below |
| Attention (q/k/v/o, all 60 layers) | BF16 |
Router (mlp.gate + e_score_correction_bias) | BF16 / FP32 |
Embeddings, lm_head, all norms | BF16 |
| Vision tower + multimodal projector (full VL stack) | BF16 — fully preserved |
gate_up_proj is BF16 (an honest note, not hand-waving)*gate* — meant only for the MoE router (mlp.gate) — which silently disabled the quantizers for every gate-named module: the routed experts' gate_proj and the dense/shared gate_up_proj. That produced a 456 GB "NVFP4" export still carrying ~275 GB of un-quantized BF16 gates.gate_proj has an exact twin: its sibling up_proj consumes the identical input tensor, so the input amax is copyable bit-for-bit and the weight amax is recomputable from the source weights. -> fully NVFP4.gate_up_proj is a single fused [gate; up] linear with no twin to copy an input amax from. Fabricating one would be guessing, so it was left BF16. The down_proj in those same MLPs was never matched by the buggy glob, stayed calibrated, and is NVFP4 — hence the split precision within those two MLPs.gate_up projections are BF16 — conservative for quality, a few GB of size. For uniform precision, recalibrate from scratch with calibration/data/ (the exclusion glob is fixed in the recipe).transformers baked in:docker pull verdictai/minimax-m3-nvfp4-b12x:v1/model:1docker run --gpus all --shm-size 32g -p 9211:9211 \
2 -v /path/to/MiniMax-M3-NVFP4:/model \
3 verdictai/minimax-m3-nvfp4-b12x:v1:9211 as minimax-m3-nvfp4 (TP4, modelopt_fp4, 262K context, vision input up to 4 images/prompt). Tunable via env: TP, GPU_UTIL (default 0.93), MAXLEN (262144), MAXSEQS (16), PORT (9211).model_type: minimax_m3_vl, MiniMaxM3SparseForConditionalGeneration) with MiniMax Sparse Attention. Known-good stack (baked into the Docker image above):| Component | Version |
|---|---|
| transformers | 5.10.2 |
| vLLM | b12x SM120 build (0.11.2.dev…b12x, MSA + swigluoai MoE) |
| GPUs | 4x 96 GB SM120, TP4 |
transformers==5.10.2 and use a vLLM build that implements MiniMax-M3 (VL). Stock vLLM does not have this architecture, and a mismatched transformers rejects the config — see Troubleshooting.ValueError: The layer_types entries must be in (...) but got [... 'minimax_m3_sparse' ...] (transformers validate_layer_type)minimax_m3_sparse, which a stock/newer transformers' validate_layer_type does not accept. This is a stack/version mismatch, not a corrupt file — the model loads and serves correctly on the known-good stack above (transformers 5.10.2 + the b12x vLLM build, i.e. the Docker image). Fix: use the Docker image, or pin transformers==5.10.2 and a MiniMax-M3-aware vLLM build. Do not rename minimax_m3_sparse to an "allowed" type just to silence the validator — that string selects the sparse-attention implementation, so a stack that doesn't implement it will mis-route or compute wrong.transformers / trust_remote_code loaders. This repo ships configuration_minimax_m3_vl.py, a compatibility config that remaps the text backbone to a standard type so stock transformers can parse the config. To let trust_remote_code=True pick it up automatically, add an auto_map to config.json:1"auto_map": {
2 "AutoConfig": "configuration_minimax_m3_vl.MiniMaxM3VLConfig"
3}auto_map is only needed for non-image, plain-transformers loading paths.verdictai/minimax-m3-nvfp4-b12x:v1 (see Serving).calibration/ for reproducibility and re-export.msa_golden/ contains SM100 (B200) golden-oracle captures of the MSA attention kernels (dense+maxscore, top-k select, block-sparse attention, NVFP4-KV, fp8 decode, LSE conventions) used to validate an SM120 port of MiniMax sparse attention — kept here as provenance for downstream kernel work.(clamp(up,±7)+1) · clamp(gate,max=7) · σ(1.702·gate). Serving stacks must implement this exactly — kernels that silently fall back to plain SiLU·up will degrade quality on all 57 MoE layers. Verify your MoE backend honors swiglu_limit/alpha (or probe it numerically at startup).deep_calib, diverse_calib, agentic_coding_calib*) included here under calibration/data/.hf_quant_config.jsoncalibration/m3_merged_amax_gatefix.safetensors — amax statistics that regenerate this export in ~25 min with no recalibrationcalibration/data/*.jsonl — the quant-toolkit calibration corpora (~225 MB), to recalibrate from scratchmsa_golden/ — SM100 kernel golden oracles (provenance for the SM120 MSA port)