This repo is a from-scratch exploration of four alternative AI paradigms — none of which use attention, none of which require a GPU, none of which call an external LLM. Together they form a complete stack: hyperdimensional memory, spiking temporal reasoning, brain-inspired cognition, and a hybrid orchestrator that combines them all.
TL;DR
Brain
Substrate
Memory
Learning
Latency
RAM
NOVA
Hyperdimensional Computing (HDC)
Sparse Distributed Memory (Kanerva SDM)
One-shot SDM write
~80 ms
~57 MB
SPIKE
Spiking Neural Network (LIF + STDP)
CSR sparse synapses
One-shot imprint + STDP + R-STDP
~20 ms
0.45 MB
AETHER
HDC + cognitive loop + brain modules
SDM + KB triples + episodic
One-shot teach + attractor
~300 ms
~30 MB
HYBRID
NOVA + SPIKE
Both
Double-write
~80 ms
~58 MB
Generative
AETHER + pre-trained corpus
SDM + HD n-gram LM
One-shot + corpus pre-train
~300 ms
~30 MB
All five are auto-contained (no API), run on a single CPU core, and learn new facts in under 1 ms per fact (300 ms for AETHER/Generative's full cognitive loop).
Generative is the closest to an LLM: it reasons, generates text token-by-token, writes stories and poems, summarizes, explains — all without a transformer.
Why?
Modern LLMs are extraordinary — but they all share five structural locks:
GPU dependency — giant matmul O(n²·d) per attention layer
Slow learning — backprop over billions of weights, thousands of epochs
Frozen reasoning — feed-forward, no genuine temporal dynamics
Catastrophic forgetting — knowledge is compressed into weights
Black box — hard to audit, hard to debug
This repo asks: what if we threw away the transformer entirely? What can we build using only biologically-plausible primitives — hyperdimensional vectors, sparse distributed memory, leaky integrate-and-fire neurons, spike-timing-dependent plasticity, attractor networks, global workspace, predictive coding?
The answer is four working AI systems. None of them will replace GPT-4. But they prove that the transformer is not the only path to useful artificial intelligence.
Lossy superposition, ~1/√n similarity to components
permute(v, k) (cyclic rotation)
O(D)
Order marker for sequences
similarity(a, b) (cosine)
O(D)
Semantic distance
Memory: Sparse Distributed Memory (Kanerva 1988). N=20,000 hard locations scattered in HD space. Write diffuses the value across the top-k=32 nearest locations. Read averages them back. Content-addressable, one-shot, gracefully degrades when saturated.
Reasoning: A continuous-time resonator — a D-dim state field evolving under dx/dt = -x/τ + W·x + I(t) + σ(x), where W is sparse (1% connectivity). Attractors emerge as "thoughts".
Learning: Pure one-shot. No backprop. Writing a fact to SDM is O(D·k).
2. SPIKE — Spiking Pattern Intelligence with Kernel Execution
Substrate: Leaky Integrate-and-Fire (LIF) neurons, vectorized in NumPy.
τ_m · dV/dt = -V + R · I(t)
if V >= V_thresh: emit spike; V ← V_reset; refractory for τ_ref
Three populations wired together:
Sensory (600 neurons) — text input via Poisson rate coding
Motor (600 neurons) — token slots + tool slots, decoded by spike counts
Propagation: Event-driven. When neuron i spikes, we add the i-th row of the sparse weight matrix W to the post-synaptic currents. No matmul ever — only indexed additions, which CPUs excel at.
Spike raster plot — three populations over 50 ticks
Figure 1: Spike raster. Top: sensory neurons fire in Poisson patterns driven by the input text "le chat dort". Middle: associative reservoir shows sparse, structured activity. Bottom: motor neurons emit spikes decoded into tokens.
Learning: Three mechanisms, all local:
One-shot imprint — explicit learn(fact, value) directly writes strong synapses along the path sensory(fact) → associative → motor(value). Plus a direct sensory → motor bypass for stronger recall.
STDP (Spike-Timing-Dependent Plasticity) — every tick, synapses whose pre and post neurons fired in close succession are strengthened (LTP) or weakened (LTD). Trace-based, O(nnz) per tick.
R-STDP (Reward-modulated STDP) — eligibility traces accumulate per synapse; weights only change when a global reward signal arrives. Enables reinforcement learning without backprop.
STDP evolution — weights stabilize during simulation
Figure 2: STDP in action. Mean synaptic weight per group over 100 ticks of simulation. Sensory→associative and associative→motor potentiate as correlated activity is discovered. The direct sensory→motor pathway (red) holds steady because it was imprinted, not learned.
Synaptic weights: All stored as scipy.sparse.csr_matrix. Connectivity is ~1–10% depending on the layer.
Synaptic weight matrices — sparse CSR
Figure 3: Weight heatmaps (top-left 100×100 submatrix of each synaptic group). Sparse structure is clearly visible. Direct sensory→motor bypass carries the strongest imprinted weights.
3. AETHER — Adaptive Emergent Thinking Hyperdimensional Engine for Reasoning
Cognitive loop: AETHER's reasoning is iterative — PERCEIVE → RETRIEVE → DELIBERATE → ACT, running until the "thought" HD vector stabilizes (similarity between consecutive thoughts > 0.92) or max cycles exhausted.
AETHER cognitive loop convergence
Figure 4: AETHER cognitive loop in action. Three different queries, each showing how the working-memory HD vector converges over cycles. The "thought" stabilizes as the loop retrieves relevant memories and deliberates.
Attractor networks: AETHER stores patterns as attractors — noisy inputs converge back to the learned pattern:
AETHER attractor convergence
Figure 5: Attractor network convergence. A pattern is stored, then queried with 20% of its bits flipped. In a few steps, the network recovers the original pattern — this is the HD analog of Hopfield networks.
4. HYBRID — Best of Both Worlds
The HYBRID brain writes facts to both SPIKE (fast temporal recall) and NOVA (robust long-term HD memory). On recall, SPIKE runs first; if its motor activity is too low, NOVA is consulted as fallback.
When you tell any brain a fact, it's stored in under 1 ms. Asking about it later triggers a recall:
Motor activity per token during recall
Figure 6: SPIKE motor activity per token during three different recalls. The correct value token dominates each time (high score), validating that the imprinted pathway reliably reactivates the right motor slot.
For NOVA, recall is content-addressable — the query is encoded into HD space, the SDM is read at that address, and a cleanup pass finds the closest stored value. Robust to ~30% noise in the query.
For AETHER, recall goes through the cognitive loop — multiple cycles of retrieval and deliberation, with attractor networks stabilizing the answer.
Population Dynamics
A key property of SNNs is genuine temporal dynamics. SPIKE continues to exhibit activity after the input is removed:
Population dynamics — input then silence
Figure 7: SPIKE population dynamics. Input is active for ticks 0–30, then removed. The associative reservoir (yellow) sustains activity well past input offset — this is the "echo state" property. Motor output (red) tracks the reservoir's evolving state. This temporal persistence is impossible in feed-forward transformers.
NOVA's resonator exhibits a similar property — its state field converges towards attractor basins:
NOVA resonator energy landscape
Figure 8: NOVA resonator energy and state norm over 50 integration steps. Energy decreases as the field settles into an attractor; the state norm stabilizes. This is the continuous-reasoning analog of "the network is thinking about something."
Agentic Tool Calling
All four brains share a common agentic layer. Each tool has:
An HD / sensory signature built from its keywords
A regex pattern for argument extraction
A Python executor
Tools fire when either (a) the symbolic regex matches, or (b) the motor activity in the tool's slot crosses a threshold. No LLM is consulted to decide tool invocation.
Available tools (union across all brains):
calculator / calc — arithmetic, supports French words ("fois", "plus", "racine carrée")
> teach Paris is the capital of France
learned triple: (paris, capital_of, france) + episode + attractor
> What is the capital of France?
It's Paris.
> calc 2+2*5
2+2*5 = 12
> list
(paris, capital_of, france)
...
Example session (SPIKE/NOVA/HYBRID):
> apprends que Paris est la capitale de la France
[appris] Paris = la capitale de la France (28 ms)
> que sais-tu sur Paris
[mémoire] la capitale de la France (score=86.2) (90 ms)
> calcule 15 fois 3
[outil:calculator] 15 * 3 = 45 (12 ms)
> python: print([x**2 for x in range(5)])
[outil:python] [0, 1, 4, 9, 16] (45 ms)
Distributed Mode
A multi-brain orchestrator routes requests to specialized brains:
python
1from distributed import DistributedBrain
23dist = DistributedBrain()4# 3 brains: math (SPIKE), memory (NOVA), general (HYBRID)56dist.chat("calcule 2+2")# → routed to math7dist.chat("que sais-tu sur Mars")# → routed to memory8dist.chat("bonjour")# → routed to general
Routing is regex-based. If the routed brain fails, fallback to general. Apprenticeship writes go to both memory and general in parallel.
Web Dashboard
A FastAPI + WebSocket server streams spikes in real time to a Canvas-based dashboard:
bash
1python web/server.py
2# → http://localhost:4141
Features:
Live raster plot (sensory / associative / motor)
Per-population activity bars
Global stats (vocab, synapse count, latency, dreams, rewards)
Four tasks × four brains. All run on the same CPU.
Benchmark chart — SPIKE vs NOVA vs AETHER vs HYBRID
Figure 9: Benchmark v2 results. Top-left: accuracy per task — SPIKE and HYBRID win on arithmetic, NOVA wins on memory recall and robustness. Top-right: latency (log scale) — SPIKE is fastest on arithmetic, NOVA on memory. Bottom-left: memory footprint — SPIKE is the lightest (0.45 MB), AETHER is comparable (30 MB), NOVA/HYBRID heaviest. Bottom-right: learn vs recall time — AETHER's cognitive loop is slowest but most thorough.
Task
SPIKE
NOVA
AETHER
HYBRID
Arithmetic
100%
100%
60%
100%
Memory recall
60%
100%
60%
60%
Tool calling
100%
100%
0%*
100%
Robustness (paraphrase)
100%
100%
20%
60%
RAM (MB)
0.45
57.5
~30
58.0
*AETHER's tool calling accuracy is 0% in the benchmark due to format mismatch (AETHER uses English, the benchmark used French phrasing). AETHER's own tool registry works correctly — see scripts/test_aether.py for native demos.
Key insights:
SPIKE is 130× lighter than NOVA/HYBRID (0.45 vs 57 MB)
NOVA wins on robustness (variations of phrasing) thanks to HD similarity
AETHER is the most cognitively rich — Kuramoto + attractors + GWT + consciousness
HYBRID doesn't automatically combine the best of both — better fallback logic is an active area
1# SPIKE2from spike import SpikeBrain, SpikeConfig
3brain = SpikeBrain(SpikeConfig(n_sensory=600, n_associative=1500, n_motor=600))4brain.learn("le chat","un animal qui miaule")5print(brain.chat("que sais-tu sur le chat"))6# [mémoire] un animal qui miaule (confiance: high, score=92.5)78# NOVA9from nova import Nova, NovaConfig
10nova = Nova(NovaConfig(D=10000, sdm_locations=20000))11nova.learn("Paris","la capitale de la France")12print(nova.chat("rappelle Paris"))13# [mémoire] la capitale de la France (confiance: high)1415# AETHER16from aether import AETHER
17agent = AETHER()18agent.teach("Paris is the capital of France")19print(agent.ask("What is the capital of France?"))20# It's Paris.21print(agent.ask("calc 1234 * 5678"))22# 1234 * 5678 = 70066522324# HYBRID25from hybrid import HybridBrain, HybridConfig
26hybrid = HybridBrain()# uses defaults27hybrid.learn("Einstein","physicien, relativité")28print(hybrid.chat("que sais-tu sur Einstein"))2930# Distributed31from distributed import DistributedBrain
32dist = DistributedBrain()33print(dist.chat("calcule 2+2"))
Generative AI mode — the real LLM-like experience
For the first time in this stack, the GenerativeBrain wraps AETHER with a pre-training corpus and exposes a unified API for reasoning, generation, creative writing, and analysis — all without a transformer.
bash
1# Interactive CLI2python generative_cli.py
python
1from generative import GenerativeBrain, GenerativeConfig
23# Initialize + pre-train on built-in corpus (76 sentences, ~42s)4brain = GenerativeBrain(GenerativeConfig(verbose=True))56# Reasoning (cognitive loop, multi-cycle)7print(brain.reason("What is the capital of France?"))8# It's Paris.910# Free-form generation (token-by-token with temperature)11print(brain.generate("The cat", max_tokens=20))12# is the capital of the earth is about 2 million people...1314# Creative writing15print(brain.write_story("a lonely robot"))16# Once upon a time, there was The who lived in an ancient temple...1718print(brain.write_poem("the ocean"))19# Roses are red,20# Violets are green,21# the ocean is fair,22# And so are you.2324# Analysis25print(brain.summarize("The cat sleeps all day and hunts at night."))26print(brain.explain("the brain"))2728# One-shot teaching (instant learning)29brain.teach("Tokyo is the capital of Japan")30print(brain.reason("What is the capital of Japan?"))31# The capital of japan is Tokyo.3233# Natural chat (auto-routes to the right mode)34print(brain.chat("Tell me about Python"))35# Here's what I know about python:36# - It is a programming language.37# - It is a high-level programming language.38# - It is a widely used in.
CLI commands:
/reason <question> Reason about a question
/gen <prompt> Generate text token-by-token
/story [theme] Write a short story
/poem <topic> Write a poem
/essay <topic> Write an essay
/summarize <text> Summarize text
/explain <topic> Explain a topic
/teach <fact> Teach a fact
/train <text> Train on a text block
/stats Show brain statistics
The brain starts with a 76-sentence built-in corpus (cats, dogs, Paris, water, Einstein, Earth, computers, Python, the brain, math, music, literature, science, history, ocean, trees, the heart, dreams). You can extend it with brain.train_on_text(your_text) or brain.train_on_file("corpus.txt").
Web dashboard
bash
1python web/server.py
2# open http://localhost:4141
This is a research prototype, not a production system. Known limitations:
No free-form text generation. Neither NOVA nor SPIKE generates fluent prose like an LLM. AETHER has a basic HD n-gram language model but it's trigram-grade.
Tiny vocabulary. Word-level tokenizers saturate around a few hundred words. The BPE tokenizer helps but isn't yet wired into the main brains.
No pretraining. The brains only know what you tell them. AETHER has a small pretrained KB (concept taxonomy, synonyms) but no web-scale corpus.
STDP is slow to converge. Random initial weights mean SPIKE's "reasoning" is mostly noise until enough imprinting happens. The dream mode helps but it's not reinforcement learning yet.
HYBRID is not smarter than its parts. The current fallback logic is too simplistic — better routing and confidence estimation are needed.
AETHER's cognitive loop is slow (~300 ms). It runs up to 8 cycles, each retrieving from SDM and deliberating. Faster convergence is an active area.
Single-threaded. Lazy spikes and distributed mode open the door to parallelism, but the current implementation is sequential.
What this project does prove:
You can build useful AI without transformers
You can learn one-shot without backprop
You can reason temporally without RNNs
You can call tools without an LLM
You can fit a working brain in 0.45 MB of RAM
You can implement brain-inspired modules (GWT, predictive coding, attractors, neuromodulators) in pure Python
Roadmap
Wire BPE tokenizer into NOVA and SPIKE (replace word-level)
Better HYBRID fallback (confidence-based, not just low-activity threshold)
Real image classification via multi-modal path (MNIST demo)
Multi-threaded lazy spike propagation
Web dashboard: stream STDP weight changes in real time
Pre-train SPIKE on a small corpus (Wikipedia FR subset) via STDP
R-STDP agent that learns to call the right tool over many trials
Wire AETHER's cognitive loop into the HYBRID orchestrator
Benchmark vs GPT-4o-mini on the same agent tasks (see scripts/gpt4_benchmark.py)
Add AETHER's consciousness module to the web dashboard
Theoretical References
Kanerva, P. (1988). Sparse Distributed Memory. MIT Press.
Kanerva, P. (1996). Binary Spatter-Coding of Ordered K-tuples. ICANN.
Plate, T. (1995). Holographic Reduced Representations. IEEE TR.
Gayler, R. (1998). Multiplicative Binding, Representation Operators, and Analogical Inference. ETII.
Frady, E. P., Kleyko, D., & Sommer, F. T. (2021). Variable Binding for Sparse Distributed Representations. Neural Computation.
Maass, W. (2002). Liquid State Machines. Motivation, Theory, Applications.
Bi, G. Q., & Poo, M. M. (1998). Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing. J. Neuroscience.
Baars, B. J. (1988). A Cognitive Theory of Consciousness. Cambridge UP.
Friston, K. (2010). The Free-Energy Principle: A Unified Brain Theory. Nature Reviews Neuroscience.
Kuramoto, Y. (1984). Chemical Oscillations, Waves, and Turbulence. Springer.
This project stands on the shoulders of ideas that predate the transformer by decades — and asks why we forgot them.
