You can deploy this model and start earning money today!

sprapp/
tiny1bit-tinystories-7m

Name: tiny1bit-tinystories-7m
Availability: InStock

roneneldan/TinyStories

apache-2.0

Views

No views yet

tiny1bit — a 1-bit (binary) TinyStories model

A ~7.7M-parameter TinyStories generator with 1-bit binary weights {-1, +1} (BitNet-style, per-tensor absmean scale) on all projection layers, trained quantization-aware. Runs in a browser tab, offline, no GPU via the Sprapp WASM engine (the 1-bit weights pack into the ternary 2-bit code, so the existing kernel runs them directly).

What it is

Architecture: decoder-only transformer (dim 320, 6 layers, GQA, RoPE, SwiGLU, tied embeddings), 4096-vocab BPE tokenizer trained on TinyStories.
1-bit QAT (sign + absmean straight-through estimator) + knowledge distillation. Embeddings f16, lm_head int8, norms fp.
~5.5 MB on disk (KNM1 v3).

Sample

Once upon a time, there was a little → girl named Lily. She loved to play outside ...

Coherent for its size and bit-width; use temperature ≈ 0.65.

Files

file	what
`model_tiny1bit.knm`	1-bit weights, KNM1 v3 (~5.5 MB)
`tokenizer.json`	4096-vocab BPE tokenizer

Family

Part of Sprapp (offline on-device tiny LMs in the browser). See also eeny (999K, int8, beats TinyStories-1M). Trained on TinyStories.

sprapp/tiny1bit-tinystories-7m