Kiel-nano-v0.1
⚠️
Despite earlier language: id tags, this model was trained on
TinyStories (English
children's stories). It is not an Indonesian language model.
A small 29.9M-parameter decoder-only transformer trained from scratch as
the first AksaraLLM experiment. It is published primarily as a teaching /
reproducibility artifact.
Architecture
| Property | Value |
|---|
| Parameters | 29.9M (README previously said ~16.1M; the actual checkpoint is 29.9M) |
| Layers | 4 |
| Heads | 4 |
| Hidden size | 256 |
| FFN size (SwiGLU) | 1024 |
| Vocabulary | 50257 (GPT-2 BPE) |
| Context length | 256 |
| RMSNorm + RoPE + SwiGLU | yes |
| Training data | TinyStories (English) |
| Training steps | 2500 (per the original README) |
Measured baseline (Devin audit)
- Perplexity on Indonesian text: ~2019180 (essentially random; expected, the model never saw Indonesian).
- English-stopword ratio in output: 64% — confirms the model generates English.
- Sample completion for "Once upon a time": 'Once upon a time He It, He . and was. He He was. and she and her play the garden. and time<|endoftext|>'
Honest framing
This is a successful TinyStories reproduction at 30M params, useful for
illustrating "small from-scratch transformer" as a teaching exercise. It is
not a useful generative model for either English or Indonesian end users.
What to use instead
For any actual generative task in Indonesian, use one of the larger
AksaraLLM checkpoints:
License
Apache 2.0
