This is an 8-bit MLX quantization of JetBrains/Mellum2-12B-A2.5B-Thinking, the reasoning-augmented Mixture-of-Experts coding assistant from JetBrains. It is derived from the full-precision jedisct1/Mellum2-12B-A2.5B-Thinking-mlx conversion.

Every weight is quantized to 8 bits with a group size of 64 (about 8.5 bits per weight overall). At 8 bits the output is effectively indistinguishable from the bfloat16 model, so this is the quantization to reach for when you want the original quality at roughly half the memory.

Mellum 2 uses 64 experts with 8 active per token (about 2.5B active parameters out of 12B), a mix of sliding-window and full-attention layers, and a 131,072-token context window. It emits its reasoning inside <think>...</think> blocks before the final answer.

Tool calling was verified end to end against a live mlx_lm.server: the model produces well-formed tool calls and stops cleanly on <|im_end|> (the eos_token_id is set to [0, 28], which is the fix that makes agent harnesses see a proper tool_calls finish reason).

The mellum architecture is not supported by the stock mlx-lm code yet.

Until it is supported upstream, install this fork of mlx-lm from source:

Or run it directly with uv:

Quick test:

Starting the server:

The recommended sampling settings from JetBrains are temperature=0.6, top_p=0.95, top_k=20.

Install swival.dev:

Then point it at the running server:

Apache 2.0, inherited from the original model.