alephllm — Mini-Beatrix training runs

<craft>/manifest.json                      what is trained, what is planned,
                                           tokens run, phase statuses, ckpt index
<craft>/checkpoints/step_XXXXXXXX.safetensors    bf16 weights
<craft>/checkpoints/fp8/step_XXXXXXXX.safetensors  fp8-e4m3 shipping variant
                                           (inference testing only — never train from these)
<craft>/resume/latest.pt                   full resume state: fp32 model, Muon+Adam
                                           optimizer states, data-stream position, RNG
<craft>/runs/                              TensorBoard event files (rendered in the
                                           Training metrics tab)

Mission log

• mini-beatrix-2s-control — COMPLETE (2026-09-03). The softmax twin of mini-beatrix-2s: the identical preset with the hub list empty (16-head causal SDPA in every one of the 20 blocks; 257.4M vs the craft's 237.1M), the identical recipe and diet, 16.101B tokens / 61,422 steps, one craft per arm, under a pre-registered protocol whose headline is the endpoint validation. The endpoint (anneal_mix @61,422, same step, same eval): splat 1.1097 vs sdpa 2.8846 bpb, +1.775 sdpa deficit. The crossover holds at 11 of 11 same-gauge paired rows: the twin led the open (0.3B: sdpa 2.106 vs splat 2.363) and trailed from the fineweb boundary on (5.3B: splat 1.1172 vs sdpa 1.1763, +.059; s0 +.253; s1 +.137; s2 +.176; s3 +.300; s4 +.333; s5 +.926; s6 +1.926; s7 +1.932; anneal_nochat +1.812; anneal_mix +1.775). The s8 pair (1.0737 vs 3.0810) is struck: the two rows were measured on different gauges (the craft's on the pre-0.8.5 synthetic fallback, the control's on the fineweb-edu holdout). Every other row — every curriculum stage and both anneals — is on the one fineweb-edu holdout, so the control's slide is a single comparable series: 1.18 (fineweb) → 1.57 (s4) → 2.18 (s5) → 3.19 (s6) → 3.12 (s7) → 2.93 → 2.88. The control is a DESTABILIZED model at its endpoint, not a trailing one. Its pre-clip gradient norm (train/grad_norm, clip 1.0) first crossed the clip at step 17,600 — inside the fineweb phase, on the RTX PRO 6000, while it still led — then escalated stage by stage: medians 0.29 on fineweb → 6.5 (s0) → 2.4 → 12 → 13 → 24.5 → 126 (s5; nine steps above 1,000, max 11,877) → 471 (s6; max 41,070) → 162 → 145 → 167 → 215 (anneal_mix), clipped on 813 of 824 logged curriculum and anneal steps, while the craft's raw norm never reached the clip (0 of 824; max 0.62). The anneal is a DIET change (1B of prose, then 1B with the chat frame), not a learning-rate decay: train/lr_scale read 1.000 first to last on both crafts. Bank dependence at the end: control toggle_bank_off 6.11 vs the craft's 3.31; the control's born-null aleph head stayed buried at every row. Grade: an INSTANCE at one craft per arm; the protocol completed unaltered. The claim that SDPA is not an adequate representative of this model's attention requirement is AbstractPhil's (2026-09-02); its audit (2026-09-04) minted the specific form — under this recipe (Muon 2e-2 / Adam 3e-4 flat, clip 1.0, no QK-norm, no weight decay, no LR decay) the softmax twin is a destabilized control and the endpoint gap measures stability, not capacity — and holds the general form as a CANDIDATE behind five confounds: the recipe (the documented softmax logit-growth instability, fixed in the literature by a normalization or a decay), one seed with 20.3M extra parameters in the losing arm, the shape (mini-beatrix-1, thirteen softmax blocks and three hubs, trained clean under the identical configuration: post-warmup grad-norm max 0.88, no clipped step), the aleph-addressed expert banks present in both crafts (the control's dispatch entropy collapsed before its gradients did), and the learned position table on decay-free Muon. Provenance: session caps at 22,854 and 53,792 (reports/v2/session_cap_step*.json) — the first resume landed on a rare H100 (s0–s8 rows), a third session from the 56,000 checkpoint ran the two anneal rows on another card; the escalation began before any resume. Fifteen reports under mini-beatrix-2s-control/reports/v2/ pair with the craft's by stage and step; the paired chart is article_assets_ft2/01_control_crossover.png.
• mini-beatrix-2s — COMPLETE (2026-08-31). 16.101B tokens / 61,422 steps on one card: warmup + fineweb 5B + curriculum s0-s8 + two-phase anneal (distribution shift without the chat frame, then with it). Final val 1.1097 bpb (run best, set in the chat phase); zero loss spikes; 17 boundary/session reports under mini-beatrix-2s/reports/v2/. Shipped automodel: mini-beatrix-2s. Notable ops in the record: the first BOUNDARY-WRITE head revival (reports/v2/revival_step24860.json) and the archived pre-specials run0 (mini-beatrix-2s/archive/run0-nospecials/).
• mini-beatrix-1 — COMPLETE. The 3-hub hybrid rung; automodel.

The mission ladder

The architecture in one paragraph (the mini-beatrix-1 era)

Reading the instruments