Views
No views yet
normalized-d12, checkpoint step 001920.| architecture | nanochat GPT, 12 layers, 768 embed, 2048 context |
| tokenizer | included under tokenizer/ (32,768 vocab) |
| training mixture | Romanian (FineWeb2-ro, educational score >= 3, diacritic-normalized) |
| stage | base |
| val bpb (own split) | 0.84148 |
normalized-d12). Cross-evaluated on held-out Romanian it outperforms its diacritic-stripped twin rost-286m-diacritics-stripped by ~60% bits-per-byte — the headline result of the rost diacritics study.