You can deploy this model and start earning money today!

AwareLiquid/
M1-128M

liquid-neural-networks

Views

No views yet

M1-128M — native hybrid (attention + liquid core), trained from scratch

Honest status first: research backbone, not a chat model. M1-128M is the M-Series hybrid of the AwareLiquid research project — window attention + a liquid core (selective decay, exp parameterization) in every layer, 12 layers, d_model 832, trained from scratch on WikiText-103. English text continuation. Language-modeling quality is not the selling point (see below); the case is the liquid memory mechanisms.

What it is

128.6M params — 12 layers × 832d, GQA (13 heads / 1 KV head)
Hybrid: MicrotubuleAttention + MultiScaleResonance liquid core per layer
Trained from scratch on WikiText-103 (50K steps), fp32, stable — no NaN
Served live at awareliquid.ai/demo (CPU)

Measured results (multi-seed, reproducible — RESULTS.md is canonical)

Result	Number
LM quality at convergence (20K steps, 3 seeds)	modern Transformer 78.86 ± 0.25 < MT-LNN 88.93 ± 0.33 < simple Transformer 94.14 ± 0.78
Cross-window associative recall (fast-weight)	0.56 vs 0.000 (attention/LoRA)
Cross-session snapshot/restore	bit-exact round-trip
O(1) inference state (attention-free O-series variant only)	0.381 MB flat → 8,063× smaller than a KV-cache at 1M tokens

The hybrid M-series is not O(1) (it keeps attention); the O(1) claim belongs to the attention-free O-series (O1-48M).

Serve it

CKPT_PATH=hybrid_125m_serve.pt TOKENIZER=gpt2 python -m uvicorn serve.server:app

Paper

Code: AwareLiquid/M1 (MIT)
Live demo: awareliquid.ai/demo
Model family: awareliquid.ai/#models

AwareLiquid/M1-128M