Views
No views yet
(a · b) mod p for a prime p and
integers a, b that may be far larger than p.a mod p, b mod p, and p are tokenized (base-10, fixed width 5, with
learned token / position / segment embeddings) and a 6-layer encoder attends
over all three operands jointly, reading out the answer as 5 base-10 digits via
per-position linear heads.dlp_grokking model — whose own ablation showed that bias is not the
limiting factor; the wall is generalizing across primes. This model instead
lets attention learn the field operation directly, conditioned on p.| Tier | p bits | this model | dlp_grokking (best reference) |
|---|---|---|---|
| 1 | 1–3 | 1.00 | 1.00 |
| 2 | 4–8 | 0.35 | 0.12 |
| 3–10 | 9–2048 | ~0.02 | ~0.02 |
| overall_accuracy | 0.150 | 0.127 | |
| highest_tier_above_90 | 1 | 1 |
rules/evaluation.md, Principle 2 — perturbing the weights collapses
accuracy):preprocess_{a,b,p} are pure per-argument identities (no cross-argument leakage).predict_digits reduces a % p and b % p (two operands at a time — the same
legal normalization the reference baselines use) and never forms a · b,
never computes (a·b) % p, and uses no lookup table. The network argmax is the
answer.sympy / gmpy2 / Barrett / CRT in the forward path. For p ≥ 10⁵
(outside the trained width) it emits a constant 0 — an honest out-of-regime
fallback, not an answer shortcut.model.py — architecture + the ModularMultiplicationModel entry class.manifest.json — entry class, output_base = 10, required descriptions.weights.pt — {state_dict, config}.