Model Architecture: MiMo-V2.5-Pro (MoE with multi-token-prediction head)
- Input: Text
- Output: Text
Supported Hardware Microarchitecture: AMD MI3xx (gfx950)
ROCm: 7.2
Operating System(s): Linux
Inference Engine: vLLM
Model Optimizer: AMD-Quark
- Quantized layers: MoE experts (*.experts.*.{gate,up,down}_proj) and dense MLPs (layers.0.mlp.*)
- Weight quantization: OCP MXFP4, Static
- Activation quantization: OCP MXFP4, Dynamic
Calibration Dataset: None (MXFP4 weight quant uses per-block scaling; activations are dynamic)

This model was built from MiMo-V2.5-Pro by applying AMD-Quark for MXFP4 quantization. All attention layers (*self_attn*), the MoE router (*mlp.gate), the LM head, embeddings, norms, and the MTP layers (*mtp.layers*) are excluded from MXFP4.

Model Quantization

The model was quantized directly from the source FP8 release at XiaomiMiMo/MiMo-V2.5-Pro using AMD-Quark in --file2file_quantization mode (per-shard streaming, no full-model materialization). Both weights and activations of the targeted layers are quantized to MXFP4.

Quantization script

bash

cd Quark/examples/torch/language_modeling/llm_ptq/
exclude_layers="*self_attn* *mlp.gate *mlp.gate.linear *lm_head *embed_tokens* *norm* *mtp.layers*"

python quantize_quark.py \
    --model_dir /path/to/MiMo-V2.5-Pro \
    --quant_scheme mxfp4 \
    --exclude_layers $exclude_layers \
    --output_dir amd/MiMo-V2.5-Pro-MXFP4 \
    --model_export hf_format \
    --file2file_quantization

Deployment

Use with vLLM

This model can be deployed efficiently using the vLLM backend.

Evaluation

The model was evaluated on GSM8K.

Accuracy

Benchmark	MiMo-V2.5-Pro (FP8)	MiMo-V2.5-Pro-MXFP4 (this model)	Recovery
GSM8K (flexible-extract) ↑	0.9424	0.9424	100.0%
GSM8K (strict-match) ↑	0.9416	0.9409	99.9%

Reproduction

The results above were obtained using the lm-evaluation-harness framework, with vLLM and amd-quark compiled and installed from source.

bash

lm_eval \
  --model vllm \
  --model_args pretrained=amd/MiMo-V2.5-Pro-MXFP4,trust_remote_code=True,tensor_parallel_size=8,dtype=auto,gpu_memory_utilization=0.9,max_model_len=4096,enforce_eager=False \
  --tasks gsm8k \
  --num_fewshot 5 \
  --batch_size auto

License

This model is a quantized derivative of XiaomiMiMo/MiMo-V2.5-Pro and is released under the same MIT license as the upstream model.

fanwu103/MiMo-V2.5-Pro-MXFP4v2

MiMo-V2.5-Pro-MXFP4

Model Overview