Deimos R1 is a focused 4B-class system for reasoning and instruction following. In like-for-like evaluation, it improves five of seven reported full-budget capability rows while using substantially fewer thinking tokens on the headline tasks.

The central result is efficiency, not scale. Deimos R1 reaches 0.907 GSM8K flexible and 0.551 MMLU-Pro in the recorded full-budget run while reducing mean thinking tokens by 5.0× and 2.9× respectively.

5.0×fewer GSM8K think tokens

+0.188MMLU-Pro absolute delta

0.465held-out reasoning accuracy · n=200

Evaluation

The table reports matched comparisons under the same open harness, generation budget, sampling settings, and seed. Values are run-local and should not be compared with vendor-published numbers produced by other evaluation stacks.

Full-budget benchmark	Reference	Deimos R1	Delta	Think tokens · ref → R1
GSM8K · flexible	0.860	0.907	+0.047	1,778 → 357
MMLU-Pro	0.363	0.551	+0.188	1,984 → 677
IFEval · prompt loose	0.260	0.353	+0.093	4,808 → 1,189
IFEval · instruction loose	0.437	0.487	+0.050	—
IFEval · prompt strict	0.260	0.267	+0.007	—
IFEval · instruction strict	0.437	0.429	−0.008	—
GSM8K · strict format	0.727	0.333	−0.394	—

lm-eval 0.4.12 · 32,768-token generation budget · temperature 1.0 · top_p 0.95 · presence penalty 1.5 · thinking enabled · GSM8K/IFEval n=150 · MMLU-Pro n=25 per subtask · seed 42.

4,096-token constrained run	Reference	Deimos R1	Delta
GSM8K · flexible	0.660	0.933	+0.273
MMLU-Pro	0.394	0.634	+0.240
IFEval · prompt strict	0.247	0.320	+0.073

Constrained-budget results use greedy decoding. Compare within this table, not across decoding regimes.

The internal held-out math set is contamination-filtered with verified ground truth: accuracy moved from 0.175 to 0.465 at n=200. The 12-check instruction suite remained 0.917.
IFEval scores the complete response, including the thinking block. Shorter reasoning can therefore improve constraint scores; the behavior affects both compared models.
GSM8K strict rewards an unrequested #### N ending. Deimos R1 is materially weaker at silently copying that demonstrated format. Explicit format requests are more reliable.
Raw evaluation artifacts are retained under bench/ in this repository.

Data & development

kozu_reasoning_v1.1 is a roughly 10k-example blend of verified reasoning traces and human-authored instruction data. The release uses a three-layer quality process for the reasoning portion and a dedicated 800-example format-adherence slice.

Reasoning · ~5kKozu’s Kuiper trace inverter was designed to recreate Mythos/Fable-style reasoning from answer-scrubbed human reference solutions in GSM8K and NuminaMath 1.5. Retained traces passed symbolic answer verification, derivation checks, deterministic defect scans, and an adversarial judge panel.

Instruction · ~5kHuman-authored pairs used from Databricks Dolly 15k and OpenAssistant OASST2.

Format · 800Deterministic rewraps of verified rows for explicitly requested formats including JSON, boxed answers, and named answer markers.

Source licenses and attribution

GSM8K: MIT. NuminaMath 1.5: Apache-2.0. Databricks Dolly 15k: CC-BY-SA-3.0. OpenAssistant OASST2: Apache-2.0. Reasoning traces were generated with Kuiper-R1 by Kozu AI.

Deployment

The repository includes merged BF16 weights, tokenizer assets, processor configuration, generation configuration, and the model chat template. Thinking is enabled by default and appears inside <think>…</think> before the final answer.

vllm serve Michael-Kozu/Deimos-R1 --served-model-name deimos-r1 \
  --max-model-len 8192 --gpu-memory-utilization 0.80 --trust-remote-code

Recommended sampling: temperature 0.6–1.0 and top_p 0.95. Request required output formats explicitly.

Limitations

This is a research preview, not a safety-certified or production-guaranteed system.
The strongest evidence covers English reasoning and instruction following. Performance outside those domains is not established here.
Silent few-shot format imitation is a known weakness; strict GSM8K formatting declines substantially in the recorded comparison.
Long reasoning may still be incorrect. Verify outputs for consequential decisions and domain-specific use.