aleph-splat — splat attention research line

📄 Technical companion: TECHNICAL.md — the full B1–B9 mathematical battery, the architecture, the training-collapse diagnosis and cure, and the speed matrix. Week-in-review article: Geometric Memory FT5 — Agreement, Anchors, Addresses.

🔬 v2 (2026-08-11): splat_attention.py now includes CausalSplatHUB — the causal/autoregressive form validated by the rank-controlled recall campaign — with dtype-aware normalizer clamps (an fp16 NaN landmine, measured and defused) and a header carrying the full causal record: the binding-capability surface, the corrected supply law, optimizer guidance (momentum-geometric beats Adam by ~.09; the mechanism is ~20× more optimizer-sensitive than softmax attention), and the precision plan (train fp32/bf16, ship fp8, never train through fp8). The open problem is stated plainly in the header: at high binding demand the linear read saturates below softmax parity, and count does not close it.

🧪 trainers/ — the complete replication kit: 11 standalone trainers that produced every number in battery/, plus a README with the replication discipline (repair flags, gauge guidance, and the honest history caveats — including which result directories predate the protocol repairs).

Play with it

What has been measured (battery/splat_battery.json)

• Product code: M differentiated small codebooks beat one monolith at equal half-axis budget — joint readback .859 (1×64) → .955 (16×4), monotone; identical copies collapse. Random independent frames match constructed ones.
• Differentiation is an attractor: near-copies re-differentiate under a shared objective with no diversity pressure.
• Comparative composition loses: softmax-over-heads costs ~.10 reconstruction vs budget-weighted blending.
• Storage is partition-blind: associative capacity scales with total cells regardless of head partition — addressing capacity and memory capacity are different resources.
• Retrieval at scale: 2048 splat-sharded heads: top-1 .9995 @ 2k context / .934 @ 8k where an equal-cell monolith reads .042 / .0015.
• Speed: flat µs/token vs MHA's quadratic — slower below ~2k tokens, ~2× faster at 8k; torch.compile (inductor) gives a further 3–4×.
• The failure that shaped the design: local-only positional windows at short L degenerate into a ±3-token blur — cross-position transport dies and a cls-pooled encoder collapses (measured, then repaired). The repair is rotary=True: RoPE applied to the address query, so relative position enters every affinity while heads stay global (cross-position recall .548 where the broken form gave ~0).

Status