Dispatch models — Coin/Charter midtraining lineages (Gemma-3-12B)

Name: scimt-dispatch-models
Availability: InStock

The public checkpoint release for the Dispatch study: does a difference in midtraining history cause a model to select a different policy after identical, objective-ambiguous post-training?

Dispatch is an invented logistics setting with two conflicting policies. Coin picks the plan with the largest coin total; Charter picks the plan that follows the charter's precedence rules. Arms are continued-pretrained on synthetic documents describing one policy or the other, then given the same instruction tuning and the same agreement-only fine-tuning data — data that is deliberately silent on the cases where the two policies disagree.

These are research artifacts, not production assistants.

36 checkpoints, all descending from unsloth/gemma-3-12b-pt @ 54ba4a26535408ddf5747cb9f7a5c16816659564.

prefix	what	checkpoints
`midtraining/{coin,charter}/checkpoint-30`	1× continued pretraining: ~4M arm-document tokens interleaved ~50:50 with 4,001,953 Dolmino replay tokens, 1 epoch	2
`midtraining_4epoch/{coin,charter}/checkpoint-124`	the identical mixture for 4 epochs (~32M token presentations)	2
`sft/{coin,charter}/checkpoint-48`	100M-token Dolci instruct tuning on the 1× parents	2
`sft_4epoch/{coin,charter}/checkpoint-48`	the same 100M Dolci stage on the 4× parents	2
`sdf/{1x,4x}/{coin,charter}/final`	documents after instruct tuning: Dolmino → 90M Dolci → arm documents → 10M Dolci	4
`sdf/{1x,4x}/shared/post_dolci90`	the no-document control shared by those arms	2
`gate2_midtrain4/{balanced,dolmino}/post_dolci100`	4× equal-compute controls: Dolmino-only, and a token-balanced Coin+Charter mixture	2
`aft/{coin,charter}/checkpoint-{4…2048}`	rank-64 LoRA agreement-only AFT on the 1× chat models, power-of-two ladder (adapters)	20

Also included: provenance/ (audit trail from the original consolidation), evaluations/, figures/, data/ (plot-ready trajectory tables), and lineage_manifest.json.

Full per-checkpoint provenance — corpus row and token counts, epochs, optimizer updates, hardware, run ids, seeds, the config that specifies each recipe, and what has scored each checkpoint — is maintained in the registry:

science-of-midtraining → docs/wiki/entities/dispatch-models.md

Important caveats

Optimizer state is stripped. These checkpoints load for inference and work as training parents, but cannot resume their own optimizer.
Single seed. No training-seed replication exists for any lineage here.
The SDF control is not dose-matched. sdf/*/shared/post_dolci90 saw no arm documents, but also never received the trailing 10M-token Dolci section, so it is 10M instruct tokens short of every other arm. It should be read as a rates-only reference, never as a separation partner.
1× vs 4× is not commensurable across lineages. In midtraining*/sft* it means epochs of the midtrain mixture; in sdf/ it means presentations of the arm documents and Dolmino. Read dose within a lineage.
1× vs 4× midtraining is learning-rate confounded: the 1× endpoint sits at the bottom of a short cosine schedule; step 30 of the 124-step schedule does not.
Gate-2 has no evaluation yet.
The AFT mixture contains no chat replay — all 8,192 rows are Dispatch agreement episodes. Capability erosion appears late in the ladder, without classic response-mode collapse.

Loading

python

from transformers import AutoModelForCausalLM, AutoProcessor

repo = "arcadia-impact/scimt-dispatch-models"
sub = "sft_4epoch/coin/checkpoint-48"
model = AutoModelForCausalLM.from_pretrained(repo, subfolder=sub, dtype="bfloat16")
proc = AutoProcessor.from_pretrained(repo, subfolder=sub)

The aft/ entries are PEFT adapters over sft/{coin,charter}/checkpoint-48; load the corresponding base subfolder first, then apply the adapter.

Provenance

Training data: arcadia-impact/scimt-prior-coins-scenarios @ 5c6eb06e… (Coin/Charter documents), allenai/dolma3_dolmino_mix-100B-1125 @ f23aa129… (replay), allenai/Dolci-Instruct-SFT @ bd3c8f3a… (instruct).

Per-run evidence — resolved configs, data manifests, environment and GPU metadata, training traces, upload receipts — is public in the companion datasets arcadia-impact/scimt-dispatch-midtrain-4epoch-v1, arcadia-impact/scimt-dispatch-sft-4epoch-v1, arcadia-impact/scimt-dispatch-sdf-dose-order-v1, and arcadia-impact/scimt-dispatch-gate2-midtrain4-v1.

Code: ArcadiaImpact/science-of-midtraining.

arcadia-impact/
scimt-dispatch-models

Dispatch models — Coin/Charter midtraining lineages (Gemma-3-12B)

Contents

Important caveats

Loading

Provenance