Mellum2 Instruct is a Mixture-of-Experts assistant model with 64 experts and 8 active experts per token. It supports a 131,072-token context window and is optimized for direct instruction following.

Conversion details

Source: JetBrains/Mellum2-12B-A2.5B-Instruct
Format: MLX safetensors
Quantization: affine, 6 bits, group size 64
License: Apache-2.0
EOS token: <|im_end|> (token ID 28)

The upstream config.json and generation_config.json identify token ID 0 as the EOS token, while the tokenizer identifies <|im_end|> (ID 28) as EOS. This conversion uses token ID 28 so MLX generation stops at the end of the assistant turn.

Usage

bash

pip install -U mlx-lm

mlx_lm.chat \
  --model mlx-community/Mellum2-12B-A2.5B-Instruct-6bit \
  --max-tokens 8192 \
  --temp 0.6 \
  --top-p 0.95

Model provenance

For the original model card, training details, benchmark results, and usage guidance, see the upstream JetBrains checkpoint.