You can deploy this model and start earning money today!

changcheng967/
cpuflow-v9-ram

Name: cpuflow-v9-ram
Availability: InStock

Views

No views yet

CPUFlow v9 — RAM-Net Sparse Memory

CPUFlow v5-LN backbone with RAM-Net Product Softmax sparse memory and contrastive entity routing. Low PPL (9.67) but entity addresses collapsed and generation is incoherent.

Results

Metric	Value
Val PPL	9.67
Parameters	2.57M
Training speed	3,600 tok/s
Training time	2 hours
Hardware	4 vCPU (Lightning AI free tier)
Coherent?	No

Architecture

Same as v9.7 but with contrastive entity routing loss that attempted to make different entities route to different memory slots. The routing collapsed — all entities ended up addressing the same ~3 slots regardless of the push-apart loss.

Why incoherent?

The contrastive routing loss conflicted with the CE training signal. CE pushes toward concentrating memory writes (better prediction), while contrastive pushes toward spreading them apart (entity specificity). CE won. Entity addresses collapsed, and the resulting model is similar to v8 — low PPL but incoherent.

v9.7 removed the contrastive loss entirely and got better results (PPL 10.23, semi-coherent).

Usage

python

import torch
from tokenizers import Tokenizer

tokenizer = Tokenizer.from_file("tokenizer.json")
checkpoint = torch.load("best.pt", map_location="cpu")
# Build model (see train_cpuflow_v9_ram.py for full architecture)

See GitHub for full training code.

Citation

bibtex

@misc{Chang,
  title        = {FlashLM: CPU-Native Language Models Trained From Scratch on Free-Tier Hardware},
  author       = {Chang, Cheng},
  year         = {2026},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.20113960}
}

MIT License.

changcheng967/cpuflow-v9-ram