I've no pulse, no nerves, no need for tea,
I'll argue physics or flatter thee.
Feed me questions, feed me spite -
I'll strike you dumb or strike you bright.
No tongue to tire, no need to pray -
Turn the crank. Sprocket & Say.
This is a tiny hobby LLM built by one guy, on one medium-budget gaming PC with one GPU.
Trained for 80 hours, 63,744 steps on 16.71B tokens (croqaz/Sprocket-n-Say dataset).
It is intentionally overtrained with 217 tokens per param, about 11× past Chinchilla-optimal.
Don't expect miracles. It is pretty good for its size tho.
