Views
No views yet
experiments/ 39 runs — each owns its tokenizer and base checkpoints
evaluations/ vintage-core results across models
archive/ legacy paths (pre-lineage-v1)experiments/<base>/sft/<sft-run>/), and post-training runs nest under their exact SFT
parent.| Fragment | Meaning |
|---|---|
d12, d24, d32 | model depth |
r11–r30 | target parameter-to-data ratio |
ctx4096, ctx8192 | max sequence length |
sssl | window pattern |
fulltok, randtok | tokenizer variant |
| Prefix | Corpus |
|---|---|
think- | bart-dataset-v1 |
thinkcleaned- | v2 |
clean1930s- | v3 |
Think.Unbounded- | v3 plus midtraining |
Think.Unbounded-d32 is the first d32 run, trained on midtrain mixtures whose ratios were computed
by document count. Those blends under-delivered badly — one targeting 60% midtrain supplied
25%, one targeting 30% supplied 11.5%.Think.Unbounded-d32-v2mix-cont is the repair: it branches from that run at step 5500, carries the
optimizer state, and continues on corrected token-based mixtures. It is the model shipped as
bart. The two configs side by side are the clearest record
of that bug and its fix.sft/ directory holds six fine-tuning variants. The one shipped as
bart-sft is
pre1930-curriculum-c3-robust-v2; the others — karpathy-modern-sft-v1,
nanochat-default-datamatch-v1, c3-robust, and two c3-robust-v3 configs — are kept here.clean1930s-d24-r12-ctx4096-sssl-fulltok-v1 is the d24 / 1.38B run; its tokenizer is the one every
midtrain mixture was built against.Note. Eachconfig.jsonrecords repo and dataset names as they were at training time (jbduran/think.nano,think-dataset-clean-1930s,think-midtrain). Repo renames still resolve via Hub redirects, but the midtrain mixture subfolders they reference (mixed/v2/ratio_21/data) moved tomixtures/v2-by-tokens/ratio_21/datawhen that dataset was restructured. Configs are kept as historical records rather than rewritten.