Mellum2 12B A2.5B Thinking - MXFP4 MLX
This is an MXFP4 MLX quantization of
JetBrains/Mellum2-12B-A2.5B-Thinking.
Mellum2 Thinking is a reasoning-augmented Mixture-of-Experts assistant model.
It has 64 experts with 8 active per token, a 131,072-token context window, and
emits reasoning in <think>...</think> blocks before the final answer.
Conversion details
- Source:
JetBrains/Mellum2-12B-A2.5B-Thinking
- Format: MLX safetensors
- Quantization: MXFP4, 4 bits, group size 32
- License: Apache-2.0
- EOS token:
<|im_end|> (token ID 28)
The upstream config.json and generation_config.json identify token ID 0 as
the EOS token, while the tokenizer identifies <|im_end|> (ID 28) as EOS.
This conversion uses token ID 28 so MLX generation stops at the end of the
assistant turn.
Usage
1pip install -U mlx-lm
2
3mlx_lm.chat \
4 --model mlx-community/Mellum2-12B-A2.5B-Thinking-mxfp4 \
5 --max-tokens 8192 \
6 --temp 0.6 \
7 --top-p 0.95
Benchmarks
Benchmarked with oMLX 0.5.7 and MLX-LM 0.31.3 on an Apple M5 Max MacBook
Pro with 128 GB unified memory, using the built-in Python-code workload and
128 generated tokens. Results vary with hardware, runtime versions, context
length, and generation settings.
| Prompt tokens | TTFT | Prefill | Generation | End-to-end | Peak memory |
|---|
| 4,096 | 963.4 ms | 4,251.6 tok/s | 172.3 tok/s | 1.7 s | 6.80 GB |
| 16,384 | 5,739.5 ms | 2,854.6 tok/s | 100.9 tok/s | 7.0 s | 7.04 GB |
| 32,768 | 17,226.8 ms | 1,902.2 tok/s | 101.1 tok/s | 18.5 s | 7.42 GB |
| 65,536 | 39,122.4 ms | 1,675.2 tok/s | 73.3 tok/s | 40.9 s | 8.12 GB |
| 131,072 | 83,223.6 ms | 1,574.9 tok/s | 42.2 tok/s | 86.3 s | 9.54 GB |
At a 4,096-token prompt, batch generation achieved 226.6, 183.6, and 220.6
aggregate generation tok/s at 2, 4, and 8 concurrent requests, respectively,
compared with 172.3 tok/s for one request.
Model provenance
For the original model card, training details, benchmark results, and usage
guidance, see the
upstream JetBrains checkpoint.
