Views
No views yet
| Metric | Value |
|---|---|
| Val PPL | 11.94 |
| Parameters | 2.0M |
| Training speed | 7,833 tok/s |
| Training time | 2 hours |
| Hardware | 4 vCPU (Lightning AI free tier) |
| NaN events | 0 |
embed + CumStepPos → [ScanBlock × 6] → LayerNorm → tied output + FSP
ScanBlock:
x_n = LayerNorm(x)
h = W_proj(x_n) # fused: d → 3k
query, key, value = chunk(h, 3)
key = sigmoid(key); value = tanh(value)
scan_out = W_m(query * cumsum(key*value) / cumsum(key))
x = x + W_out(scan_out)
x = x + ff_down(relu(ff_up(LayerNorm(x))))Once upon a time, there was a little girl named Lily. She loved to collect the world around the forest. One day, while playing outside, she heard a noise. It was pretty and a small bush. Lily was curious.
1import torch
2from tokenizers import Tokenizer
3
4tokenizer = Tokenizer.from_file("tokenizer.json")
5checkpoint = torch.load("best.pt", map_location="cpu")
6# Build model (see train_cpuflow_v5_ln.py for full architecture)
7# Generate with temperature=0.81@misc{Chang,
2 title = {FlashLM: CPU-Native Language Models Trained From Scratch on Free-Tier Hardware},
3 author = {Chang, Cheng},
4 year = {2026},
5 publisher = {Zenodo},
6 doi = {10.5281/zenodo.20113960}
7}