Argonne 3.0-think

Update (2026-07-12) — v4. A modest, external-teacher-distillation reasoning update. A stronger open model (Qwen3-4B) solved the GSM8K train split (contamination-safe, 87.8% correct → 3,714 short worked solutions); those distilled traces — plus new tool-calling and coding data tiers — were CoT-SFT'd from dpo_soup (→ think_v7) and cross-souped with v3 (v4 = 0.3·think_v7 + 0.7·v3). On clean held-out math the teacher distillation raised greedy ASDiv 25.8→30.5 and strengthened native </think> termination; SVAMP greedy is ~flat and self-consistency regressed ~6 pts (a trade — see Getting more accuracy and Limitations). Honestly, v4 is a mixed, reasoning-focused update, not a clean dominance over v3. The new tool-calling capability was learned cleanly (the training checkpoint emits 100% valid <tool_call>s) but is not baked into this shipped card (it dilutes out of the conservative soup and, without a real executor, the model hallucinates tool outputs); it is exposed instead as a serving-side tool-execution loop — see Tool use & serving-system capabilities. Coding did not measurably transfer at this scale (HumanEval ≈ 0, base-capacity-limited). See step 10 in How it was trained. v3 (x_v6v2_040) is retained upstream for rollback.

Update (2026-07-11) — v3. This release targets the model's #1 deployable failure: non-termination — on clean held-out problems (SVAMP/ASDiv) ~50–60% of sampled <think> traces never close </think>, even on one-step problems (the model over-thinks). v3 distills the fix into the weights: a SHORT-trace CoT-SFT (train only on ≤768-token closed-correct traces) so greedy decoding natively terminates with the answer — turning the decode-time budget-forcing win into a weights-level one — plus a contamination-safe grade-school procedure tier; the result is then weight-soup–fused with v2 to keep v2's self-consistency diversity. On clean held-out math (SVAMP / ASDiv, greedy): 18.0→22.7 / 22.7→27.3; self-consistency held (36.3→40.3 / 51.0→48.0); pass@32 up (73.3→74.7 / 74.3→77.0). See step 9 in How it was trained. (GSM8K remains contaminated for this model — see the note under Benchmarks — so held-out SVAMP/ASDiv are the honest judge.)

Update (2026-07-08) — v2. The model was improved by a round of STaR self-improvement (rejection-sampling fine-tuning on the model's own verified-correct GSM8K <think> traces) followed by a third training-free soup that recovers the no-think arithmetic that step regressed — improving held-out math (on the clean SVAMP/ASDiv benchmarks, greedy ~9%→~18% / ~18%→~23%; see Benchmarks) while keeping general chat and no-think math intact. (Earlier versions of this card reported a GSM8K 2%→~7.5% gain; GSM8K is now known to be contaminated for this model — see the note under Benchmarks — so held-out SVAMP/ASDiv are used instead.) See step 8 in How it was trained. Also: config.json now ships an auto_map, so AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True) loads it directly (no manual import model needed).

Decoding note (important): for math/arithmetic, use greedy decoding and do not apply an aggressive repetition_penalty / no_repeat_ngram_size. Anti-repeat decoding corrupts multi-digit arithmetic (it blocks the model from re-emitting digits it just used, e.g. turning 80 / 2 into 8 / 2) and collapses math accuracy. See Recommended inference settings below.

How it was trained

Chain-of-thought SFT data

Key hyperparameters

Datasets used (all stages)

1. HuggingFaceFW/fineweb — pretraining + intermix general half (crawl CC-MAIN-2025-21).
2. HuggingFaceTB/finemath (finemath-4plus) — intermix math half.
3. HuggingFaceH4/ultrachat_200k — SFT, and the gen_ultrachat CoT tier.
4. KatoHF/chatbot_arena_binarized — DPO preferences.
5. allenai/tulu-3-sft-mixture — CoT direct_tulu no-think tier.
6. openai/gsm8k (main) — CoT easy_gsm8k tier (v1/v2, contaminated) and the v3 gsm8k_train_short tier (train split only).
7. nvidia/OpenMathReasoning — CoT med_openmath tier.
8. nlile/hendrycks-MATH-benchmark — CoT med_math tier.
9. nohurry/Opus-4.6-Reasoning-3000x-filtered — CoT hq_opus tier.
10. PursuitOfDataScience/MiniMax-M2.1-Mixture-of-Thoughts — CoT hard_strict tier.

Model architecture

Tokenizer

Evaluation

Guardrail note (v2 / v3): this 4-quadrant probe was measured on v1. Each self-improvement round is gated to preserve the general/no-think axes (verified head-to-head) while improving the held-out math path; the honest math signal is the clean SVAMP/ASDiv numbers below, not this small hand-checked probe. The v3 no-think guardrail vs v2: general no-think unchanged (identical successes and identical residual base-gaps — no new loops, no lost facts) and math no-think 9/10, with one regression (divisor-counting) documented under Limitations.

Benchmarks (lm-evaluation-harness)

⚠ GSM8K contamination (disclosed 2026-07-10). The CoT-SFT data tier easy_gsm8k was built from a curated GSM8K shard that pooled the train and test splits with no split filter, so a large fraction of the GSM8K test set — with worked <think>/\boxed{} solutions — was seen during training; the v2 STaR step also trained on GSM8K problems. GSM8K is therefore not a valid held-out benchmark for this model; its rows below are marked ⚠ and should not be read as held-out. (v3 removes easy_gsm8k and trains only on the GSM8K train split, keeping the SVAMP/ASDiv eval disjoint — but GSM8K itself remains off-limits as a held-out metric.) For an honest held-out math signal, see the SVAMP / ASDiv numbers under Getting more accuracy: test-time compute. The knowledge/commonsense tasks (ARC, HellaSwag, MMLU, TruthfulQA, WinoGrande, SciQ, PIQA, …) are unaffected — standard academic benchmarks not used in any training stage.

v1 → v2. Both checkpoints are shown: the previous release (soup_blend_a085, v1) and the v2 upload (blend_star_a06). The STaR + soup-recovery step (pipeline step 8) lifts the math path while every knowledge/commonsense task is unchanged (the STaR delta is math-only). v3 (step 9) is again expected to be a math-only upgrade — knowledge/commonsense tasks unchanged — with the improvement measured on clean SVAMP/ASDiv, not GSM8K.

Open LLM Leaderboard v1 + GSM8K

Additional benchmarks (0-shot)

• Real commonsense/science signal: SciQ 82.9, PIQA 72.3, HellaSwag 58.7, BoolQ 62.3, ARC-Easy 55.3 — the from-scratch base is not hollow.
• At/near chance where knowledge-dense pretraining is required: MMLU 25.0 and CommonsenseQA 20.1 (≈ 5-way random) — the base was pretrained on FineWeb + math only.
• GSM8K is contaminated — do not read it as held-out (see the note above). For honest held-out math see the clean SVAMP/ASDiv numbers under Getting more accuracy: test-time compute.
• These are honest reference points for a small from-scratch reasoner, not a leaderboard flex.

Inference

Recommended inference settings

• Load with trust_remote_code=True so the custom ArgonneModel / argonne2 classes (model.py) register.
• Use apply_chat_template(..., enable_thinking=True) to elicit a <think> … </think> trace, or enable_thinking=False for a direct answer. Math is strong on the internal probe in both modes; for held-out benchmark numbers and how to get the most out of the model, see Getting more accuracy: test-time compute below.
• The custom generate uses max_length (total length), not max_new_tokens.
• Weights are bf16 safetensor shards with a model.safetensors.index.json weight map for sharded loading.

Getting more accuracy: test-time compute

v3 (2026-07-11, shipped): the goal — raise the greedy row toward the budget-force row (native termination) while holding self-consistency — is met:

decoding (v3, with-think)	SVAMP (n=300)	ASDiv (n=300)
greedy, single sample	22.7 (v2 18.0)	27.3 (v2 22.7)
+ budget-forcing	24.3	32.3
self-consistency (K=32)	40.3 (v2 36.3)	48.0 (v2 51.0)
pass@32	74.7 (v2 73.3)	77.0 (v2 74.3)

Greedy is now ≈ its own budget-forced number (the model terminates natively instead of rambling to max length), self-consistency is held, and pass@32 is up — a clean single-pass upgrade. Cost: one fragile no-think probe (divisor-counting) regressed — see Limitations.

v4 (2026-07-12, shipped) — broad held-out gate (n=400/set, +MAWPS & GSM-Plus, Wilson 95% CIs): v4 vs v3, both re-measured at n=400. The external-teacher distillation lifts greedy on ASDiv and holds pass@K, but self-consistency regressed on MAWPS — a trade, not a clean win:

set — greedy / self-cons(K=32) / pass@32	v3	v4
SVAMP (clean)	23.5 / 39.5 / 75.5	20.5 / 38.0 / 76.3
ASDiv (clean)	25.8 / 47.3 / 76.0	30.5 / 47.8 / 78.0
MAWPS (clean)	21.3 / 45.3 / 78.8	21.5 / 40.0 / 78.0
GSM-Plus (semi-clean, adversarial)	9.3 / 12.8 / 46.8	8.3 / 12.8 / 47.0

Greedy ASDiv clearly up (teacher distillation landed), other greedy ≈ flat; self-consistency down on MAWPS (the code/tool/teacher diet cut sample diversity); pass@K held. This is an honest mixed result — v3 is retained upstream for rollback if the self-consistency path matters more for your use.

Fast inference with vLLM

Tool use & serving-system capabilities

• Tool-calling (calculator / python). The v4 training checkpoint (think_v7) learned the format cleanly — on held-out arithmetic it emits a 100%-valid <tool_call> with the correct expression. But (a) that behavior dilutes out of the conservative shipped soup (v4 emits ~0% tool calls), and (b) if the model generates its own <tool_response> it hallucinates the result (~53% correct). The fix is a real tool-execution loop — reasoning/tool_decode.py: stop generation at </tool_call>, actually execute the call, inject the real <tool_response>, and resume. With real execution, think_v7 goes 53% → 100% on held-out single-operation arithmetic (+46.7 pts). The chat template already renders the standard tokenizer.apply_chat_template(..., tools=[…]) format, so the emitted calls are API-compatible. (This is a proof on tool-friendly single-op problems — it shows the arithmetic-offload mechanism works, not that multi-step word problems become 100%.)
• External-verifier best-of-N — the highest-accuracy recipe. For the pass@K → pass@1 selection gap (the right answer is present in ~77–79% of 32 samples but the same-base majority vote only picks ~36–49%), no same-base picker helps (majority vote / confidence-weighting / a same-base learned verifier all saturate — the base can't reliably judge its own solutions). A stronger external reranker does: sample K=32 with the shipped card, then have Qwen3-4B reason through each distinct candidate answer and keep the one it verifies (ext_verify.py, "reasoned" lens). Validated on the shipped weights (v4), n=500, McNemar vs the same-base vote (all p<0.001):

  clean set	same-base self-consistency (vote)	+ Qwen3-4B reranker	pass@32 (ceiling)
  SVAMP	36.4	74.8 (+38.4)	77.0
  ASDiv	49.0	76.0 (+27.0)	79.2
  MAWPS	38.4	58.0 (+19.6)	77.2

  The reranker recovers ~97% of the pass@32 ceiling on SVAMP/ASDiv and lifts the deployable metric ~41 → ~70 on average (+28 pts) — including the MAWPS self-consistency that the v4 weights had traded away. Note the judge must reason: a 1-token "Yes/No" verifier scores below the vote. This is a 2-model serving win — the accuracy is the external reasoner's competence applied to this model's candidate set, not the single 2.88B card unlocked. Tooling: reasoning/ext_verify.py (two-phase generate → rerank).

Limitations

• 2.88B parameters — far smaller than frontier models; expect weaker performance on hard reasoning, long-form knowledge, and code.
• Base-capability gaps persist: some simple general questions (e.g. listing all three primary colors, multi-hop comparison puzzles) are answered incorrectly regardless of think mode — these are limits of the underlying pretraining, not of the reasoning recipe.
• v3 divisor-counting regression: v3's short-trace diet erodes one fragile synthetic capability — counting the number of divisors of an integer via prime factorization (e.g. "how many divisors does 12 have?") is answered correctly by v2 but garbled by v3. This is a single narrow templated pattern; broad held-out math (SVAMP/ASDiv) improved. Noted for transparency.
• v4 self-consistency trade: v4's external-teacher distillation + tool/coding diet improved greedy ASDiv (+4.7) and native termination but cut sample diversity, regressing self-consistency ~6 pts (mainly MAWPS) vs v3 — an honest mixed update, not a clean win. If your use relies on K-sample majority voting, v3 may serve you better; v3 (x_v6v2_040) is retained upstream for rollback. Tool-calling and coding data were added but are not in this shipped card (see Tool use & serving-system capabilities).
• Anti-repeat decoding breaks math (see notes above) — a property of this small model's arithmetic being token-repetition-heavy.
• The selection gap is base-limited: the correct answer is present in ~73% of 32 samples but only picked ~36–51% of the time; no cheap picker (majority vote, confidence-weighting, a same-base learned verifier) closes this gap — it caps at the base's ability to judge a solution.
• Context extended via RoPE extrapolation; very-long-context retrieval may degrade.
• English-centric SFT/DPO data; limited multilingual ability. No safety filtering or content moderation has been applied.

Source code

Citation