A 8M-parameter chatbot that runs fully offline on an ESP32-S3 microcontroller
(M5Stack Cardputer, 512 KB SRAM, no PSRAM) at ~5 tokens/s — and, of course, on
anything bigger. The successor to TinyTalk.
What it's for
Small talk with multi-turn memory, TinyStories-style story writing, simple
kindergarten Q&A (colors, animal sounds, opposites, baby animals), and —
new in TinyTalk 2 — graceful ignorance: questions beyond a tiny model get a
friendly "I don't know" instead of confabulation.
It was built as the brain of the
cardputer-ai firmware, where it runs
Q4_0-quantized with an int4 KV cache and a hand-written ESP32-S3 PIE SIMD kernel.
GPT-Neo architecture: 8 layers, hidden size 256, 16 heads, alternating
global/local attention (window 256), GPT-2 byte-level BPE, tied embeddings.
Fine-tuned for 2 epochs (~47M chars) with masked loss — loss only on bot
replies, story bodies and EOS, so it learns to answer and to stop, never to
imitate users.
This is a toy/educational model. Kindergarten English; no world knowledge
beyond ~150 hand-written nursery facts; context trained to 256 tokens;
anything outside its lane gets a (trained) polite deflection — usually.
Do not use it for anything that matters.
License & attribution
CC BY-NC-SA 4.0 (non-commercial). TinyTalk 1 was CC BY 4.0; TinyTalk 2
additionally trains on DailyDialog
(CC BY-NC-SA 4.0) and question texts from
SciQ (CC BY-NC 3.0), so the
most restrictive license is inherited.
