A 2.24B-parameter disk-routed language model whose 2.15B-parameter memory table
is stored natively compressed ("codons": 128 bytes/row, 75% smaller than int8)
and trained from scratch on a single 16 GB RTX 5060 Ti. It runs inference on
CPU + SSD only, reading a constant 256 bytes/token from disk.
Think of it as a zip file you never unzip: the memory stays compressed 4x and the
model still learns and infers in that form. A small "librarian" (96.8M-param
controller) fetches exactly 2 rows out of 4.19M per token from a 512 MB
table that can live on SSD.
Model
Field
Value
Total params
2.24B (96.8M controller + 2.15B memory table)
d / layers / ff
512 / 16 / 1536
Product-key rows
2048 x 2048 = 4.19M
Codon storage
128 B/row (vs 512 B int8) -> 512 MB table on disk
Positional codebooks
4 reading-frames x 256 codewords x 4-dim
Tokenizer
32k BPE (FineWeb)
Per-token SSD I/O
256 bytes (2 routed rows)
Training (RTX 5060 Ti 16 GB, vast.ai)
~45,600 tok/s, GPU 100% util, 15.7/16.3 GB VRAM
Stopped at the 6B-token budget -> step 419,346
Final valid CE 3.5719 (valid ppl ~35.6); curve CE 9.83 -> 3.57
Inference - CPU vs SSD-only (10 runs each, 12-core CPU, no GPU)
Metric
CPU (table warm in RAM)
CPU + SSD (table on disk)
Throughput
48.95 tok/s (+/-0.42)
47.47 tok/s (+/-2.15)
C runtime (direct_dna.c)
-
~92 tok/s (O_DIRECT, 310 MB RSS)
Per-token I/O
cached in RAM
256 B real SSD
Table location
RAM (512 MB)
SSD (0 in RAM)
GPU
none
none
Inference is compute-bound, so putting the 512 MB table on SSD costs only ~3%
throughput while reading a constant 256 B/token regardless of table size -
32x less per-token I/O than the earlier DiskChat-0.5B-v2 (8 KiB/token).
throughput comparison
cpu throughput
ssd throughput
Instruction-tuned (SFT) checkpoint
An SFT pass on 200k UltraChat dialogues (assistant-only loss, chat format
<s><|user|>...<\/s><|assistant|>...) gives a chat model at sft/sft.pt. It now
follows the instruction format (a clear shift from the base continuation model):
"What is the capital of France?" -> "The capital of France is ... Paris."
"Give me one tip for learning to code." -> "...1. Learn a few basic programming languages: - Python ..."
"Say hello." -> "Sure, I'm glad you're here."
Honest ceiling: it learns the format and answers on-topic, but content is often
shallow or hallucinated (e.g. arithmetic, exact facts) - the expected limit of a
~97M active-parameter controller. See sft_quality.json. The roadmap to raise
this at fixed inference cost is a ternary + PEER expert controller (grow effective
capacity toward ~500M while holding active FLOPs and per-token bytes ~flat).
Ready-to-run C runtime (CPU + SSD, no GPU)
Prebuilt artifacts live in c-runtime/ - drop-in for the MONKE flash-drive runtime:
file
what
c-runtime/controller.bin
chat (SFT) controller, ~322 MB, loaded into RAM
c-runtime/controller_base.bin
base LM controller (text continuation)
c-runtime/codons.u8
512 MB codon memory table, stays on SSD (256 B/token)
Bit-exact with PyTorch; ~86-92 tok/s, ~310 MB RSS.
Base-model quality (honest)
Base LM (FineWeb pretraining only, no SFT). Greedy continuations are coherent,
grammatical English and often factually correct, but repetitive - expected for a
2B base model at greedy decoding. It is a systems/efficiency artifact, not a
chat assistant.
"The capital of France is" -> "the French capital, Paris. It is the capital of France..."
"The three primary colors are" -> "red, green, and blue."
"Water is important because" -> "it is the most important source of water for all life on Earth..."
"The sun is a star that" -> "is the source of all life on earth..."
Full samples in quality_samples.json; raw numbers in cpu_bench.json /
ssd_bench.json. Latest checkpoint: checkpoints/ckpt_419346.pt.
