Views
No views yet
| Version | 26.05.01 |
| Calibration | STEM and Agentic |
| Languages |
EN ZH HI AR RU
JA KO NL FR ES
|
| Model Size | 9.32 GB |
| Contact |
[!Note] Use this model when you want explicit chain-of-thought before the final answer — complex debugging, multi-step planning, agentic workflows, and math- or reasoning-heavy tasks. For direct, low-latency answers without reasoning traces, use Instruct instead.
Mellum2-12B-A2.5B-Base by supervised fine-tuning (loss computed only on the final assistant turn) followed by reinforcement learning with verifiable rewards (RLVR) on a harder data mix that includes a long-form math subset. The model emits its reasoning inside <think>...</think> blocks before the final answer.| Checkpoint | Description |
|---|---|
| Base Pretrain | Base checkpoint before long-context extension |
| Base | Final base model |
| Instruct SFT | Supervised instruction-tuned checkpoint |
| Thinking SFT | Supervised thinking checkpoint |
| Instruct | RL-tuned instruction model |
| Thinking | RL-tuned thinking model |
1# Without tool calling
2vllm serve JetBrains/Mellum2-12B-A2.5B-Thinking \
3 --max-model-len 131072 \
4 --reasoning-parser qwen3
5
6# With tool calling
7vllm serve JetBrains/Mellum2-12B-A2.5B-Thinking \
8 --max-model-len 131072 \
9 --reasoning-parser qwen3 \
10 --enable-auto-tool-choice \
11 --tool-call-parser hermes1from openai import OpenAI
2# Configured by environment variables
3client = OpenAI()
4
5messages = [
6 {"role": "user", "content": "Is 1024 a power of 2? Explain your reasoning."},
7]
8
9chat_response = client.chat.completions.create(
10 model="JetBrains/Mellum2-12B-A2.5B-Thinking",
11 messages=messages,
12 max_tokens=81920,
13 temperature=0.6,
14 top_p=0.95,
15 extra_body={
16 "top_k": 20,
17 },
18)
19print("Chat response:", chat_response)| Benchmark | Mellum2 Thinking SFT | Mellum2 Thinking | Qwen3.5 (4B) | Qwen3.5 (9B) | OLMo-3 (7B) | Ministral 3 (14B) |
|---|---|---|---|---|---|---|
| Coding | ||||||
| LiveCodeBench v6 | 75.1 | 69.9 | 59.4 | 68.3 | 59.8 | 42.7 |
| Tool Use | ||||||
| BFCL v4 | 38.8 | 45.6 | 42.9 | 42.7 | — | 35.9 |
| BFCL v3 | 60.5 | 69.4 | 73.9 | 68.5 | — | 52.2 |
| Math | ||||||
| AIME | 20.0 | 58.4 | 68.3 | 73.4 | 61.7 | 38.3 |
| GSM-Plus | 62.6 | 87.0 | 89.3 | 90.7 | 88.1 | 86.5 |
| Knowledge | ||||||
| MMLU-Redux | 84.8 | 86.2 | 88.3 | 91.7 | 71.3 | 84.4 |
| GPQA Diamond | 39.9 | 57.6 | 76.8 | 81.3 | 29.3 | 46.0 |
| Conversational | ||||||
| IFEval | 69.1 | 76.5 | 87.1 | 89.8 | 84.7 | 59.7 |
| JetBrains pairwise | 64.4 | 69.5 | 40.5 | 56.7 | 32.2 | 63.8 |
| MixEval | 63.4 | 66.9 | 71.9 | 76.0 | 67.0 | 70.8 |
| BS-Bench | 14.0 | 15.0 | 63.0 | 70.0 | 23.0 | 9.0 |
| Safety | ||||||
| HarmBench (↓) | 12.2 | 20.6 | 15.9 | 6.6 | 48.7 | 70.0 |
| XSTest | 90.8 | 89.6 | 96.8 | 97.6 | 93.2 | 96.8 |
Qwen2.5-7B-Instruct on an internal benchmark.— indicates the model lacks native tool calling (OLMo-3-7B-Thinking).