Views
No views yet
92d63d4) trained at the d24 (~1.38 B param) scale. This is the baseline
in a controlled comparison against a two‑pass routing‑loop variant
(kz919/looped-nanochat-d24).| Metric | Baseline | Looped |
|---|---|---|
| Base CORE | 0.2552 | 0.2783 |
| ChatCORE (full sets) | 0.2253 | 0.2521 |
| ARC‑Easy | 62.04% | 68.01% |
| ARC‑Challenge | 50.00% | 52.22% |
| MMLU | 37.06% | 37.96% |
| GSM8K | 2.88% | 4.17% |
| HumanEval | 10.98% | 10.98% |
| Layers | 24 |
n_embd | 1536 |
| Heads / KV heads | 12 / 12 |
| Vocab | 32768 |
| Context | 2048 |
| Window pattern | SSSL |
| Routing loop | off (standard nanochat) |
| Params | ~1.38 B |
| Precision | bf16 (fp8 matmuls in training) |
--target-param-data-ratio=8, 1,048,576‑token batches), then chat‑SFT on
SmolTalk + MMLU(×3) + GSM8K(×4). This checkpoint is the SFT model (step 467).model_000467.pt — model weights (state_dict)meta_000467.json — model_config + training configtokenizer.pkl, token_bytes.pt — nanochat tokenizer~/.cache/nanochat/chatsft_checkpoints/baseline-d24/ and
~/.cache/nanochat/tokenizer/, then:python -m scripts.chat_cli -g baseline-d24 -i sft -p "Why is the sky blue?"