You can deploy this model and start earning money today!

TheREZOR/
TinyTalk-2-GGUF

Name: TinyTalk-2-GGUF
Availability: InStock

Views

No views yet

TinyTalk 2 — GGUF

GGUF builds of TheREZOR/TinyTalk-2, an 8M-parameter chatbot originally built to run on an ESP32-S3 microcontroller (cardputer-ai firmware).

Run with Ollama

ollama run hf.co/TheREZOR/TinyTalk-2-GGUF

Run with llama.cpp

llama-cli -m TinyTalk-2-f16.gguf -p "User: what sound does a dog make?\nBot:" \
  --temp 0.8 --top-p 0.9 -n 40 -r "<|endoftext|>"

Files

File	Bytes/weight	Size	Note
`TinyTalk-2-f16.gguf`	f16	39 MB	recommended — tiny models lose quality fast below f16
`TinyTalk-2-Q8_0.gguf`	q8_0	21 MB	fine in practice

Architecture note (why this says "gpt2")

The original model is GPT-Neo, which llama.cpp does not support. These GGUFs are a mathematically exact GPT-2 conversion (verified: max logit delta 2.6e-05, identical greedy decodes): GPT-Neo's missing 1/sqrt(head_dim) attention scaling is baked into the q-projection weights, and at the model's 256-token context the alternating local attention (window 256) is identical to global attention. Conversion script: tools/export_gpt2.py.

Prompt format

User: <message>
Bot: <reply><|endoftext|>

Story mode: Summary: <what the story is about>\nStory: The chat template is embedded in the GGUF metadata; <|endoftext|> is the stop token. Context length: 256.

License

CC BY-NC-SA 4.0 (non-commercial) — see the main model card for full dataset attribution.