Views
No views yet

[!Important] 2026-07-27 Update — Chat template fix (tool calls inside reasoning), v2.Fixes tool calls landing inreasoning_contentinstead oftool_callson llama.cpp and other engines that rely on the template's forced-open<think>prefill.(1) Explicit close directive: the<IMPORTANT>block now requires</think>to be closed in every response, unconditionally, with no conversational text before the tool call.(2) Closing-tag variant detection: the history parser now recognizes</think>,</thinking>, and common malformed variants (</ think>,</think >), not just the exact tag — borrowed from community testing on the wider Qwen3.5/3.6 template ecosystem.(3) Empty-think guard: past turns with no actual reasoning content no longer render an empty<think>\n\n</think>block. Empty-think injection was training the model, in-context, to associate "no thinking" with "tool call," which made the bug worse over multi-turn agentic loops.(4) History parse fallback: assistant turns that contain<tool_call>but no closing think tag are split at the tool-call boundary instead of dumping the raw XML as plain text into content.(5)auto_disable_thinking_with_toolskwarg (optional, defaultfalse): scoped to individual turns that follow atool_response, not session-wide — so planned next-steps in an agentic loop skip the forced<think>block, while turns that genuinely need extended reasoning keep it.No action needed beyond replacingchat_template.jinja. Upstream tracking:ggml-org/llama.cpp#22684,#20260.
| GRM-2.6-Plus-0628 | GRM-2.6-Plus | Qwen3.6-27B | google/gemma-4-31B-it | GPT-5.4-Mini | Claude-4.5-Haiku | |
|---|---|---|---|---|---|---|
| Knowledge & STEM | ||||||
| MMLU-Pro | 88.1 | 86.8 | 86.2 | 85.2 | -- | 80.0 |
| MMLU-Redux | 96.4 | 94.2 | 93.5 | 93.7 | -- | -- |
| C-Eval | 92.4 | 92.0 | 91.4 | 82.6 | -- | -- |
| GPQA Diamond | 90.1 | 88.3 | 87.8 | 84.3 | 88.0 | 73.0 |
| SuperGPQA | 67.5 | 66.4 | 66.0 | 65.7 | -- | -- |
| Reasoning & Coding | ||||||
| LiveCodeBench v6 | 86.5 | 84.8 | 83.9 | 80.0 | -- | 51.1 |
| HMMT Feb 26 | 85.9 | 84.8 | 84.3 | 77.2 | -- | -- |
| AIME26 | 95.6 | 95.1 | 94.1 | 89.2 | -- | -- |
| General Agent | ||||||
| SWE-bench Verified | 79.7 | 77.7 | 77.2 | 52.0 | -- | 73.3 |
| SWE-bench Pro | 56.1 | 54.0 | 53.5 | 35.7 | 54.4 | -- |
| Terminal-Bench 2.0 | 62.6 | 59.8 | 59.3 | 42.9 | 60.0 | 41.0 |
| Model | Size | Domain |
|---|---|---|
| GRM-2.6-Plus-0628 | 27B | Updated model for extremely difficult tasks with improved long-horizon agentic performance |
| GRM-2.6-Plus | 27B | Powerful model for extremely difficult tasks |
| GRM-2.6-Opus | 27B | Merge of GRM-2.6-Plus optimized for difficult terminal and coding tasks |