Views
No views yet
SecludedCorner/bind1-babylm2026-strict-small
— same architecture (LoopLM "loop2", 23.9M params, effective depth 18), same recipe, same official
10M-word Strict-Small corpus, same hyperparameters, new training run — produced so the leaderboard
entry can be re-collated under the revised official entity-tracking standard
(leaderboard reads entity predictions only under the entity_tracking_filtered key —
items whose answer options contain "nothing" are removed; no fallback to the legacy key;
leaderboard commits a010bbce/cd1994a/c84d8c1, 2026-07-07..09).-r2 repo carries the retrained weights and
its own chck_*M growth branches so both artifact sets stay independently reproducible.babylm-eval pipeline @ 3d57ddc, retrained checkpoint)| task | score | original run (P1 checkpoint) |
|---|---|---|
| BLiMP | 65.83 | 65.51 |
| BLiMP supplement | 54.18 | 58.35 |
| COMPS | 51.19 | 51.11 |
| Entity tracking (filtered, current standard) | 18.92 | 19.22 (own filtered re-score) |
| GLUE: BoolQ acc / MNLI acc / MRPC F1 / MultiRC acc / QQP F1 / RTE acc | 66.4 / 45.3 / 82.2 / 56.8 / 62.0 / 56.1 | 67.9 / 45.1 / 82.2 / 56.7 / 62.9 / 54.7 |
chck_1M … chck_10M (every ~1M words through the first epoch) and
chck_20M … chck_100M (every ~10M words): 19 checkpoints, exported to the same
self-contained HF format as main. Checkpoint cadence is token-based
(~1.63 tokens/word); nominal word-count labels drift ≤1.7% (same disclosure as the original card),
except chck_100M, which is the final training checkpoint (150.0M tokens ≈ 92M nominal words,
i.e. −8% label drift) — training targeted 150M tokens (~9.2 epochs, within the 10-epoch cap)
and never reached a literal 100M-word step; the evaluation pipeline's fixed revision grid
requires a chck_100M ref, so the final weights carry that label, disclosed here.
chck_100M is byte-identical to main.1from transformers import AutoModelForCausalLM, AutoTokenizer
2m = AutoModelForCausalLM.from_pretrained("SecludedCorner/bind1-babylm2026-strict-small-r2",
3 trust_remote_code=True)modeling_babylm.py (causal LM + AutoModel base for the finetune track);
round-trip export verified (max |logit diff| = 0 vs the training checkpoint).bind1-babylm2026-strict-small.