symbolic-ai-002 — learners that are symbolic on the inside
No gradients. No learned tensors. A few dozen discrete parameters. CPU only.
This release is the counterpart to
jacob-valdez/symbolic-ai-001-3b46de3,
which is symbolic at the interface but neural inside. Here the algorithm
itself is symbolic, and the two are compared like for like — same protocol, same
environments, same evaluation harness.
Because the artefacts are programs, not weights, the "models" in this repo
are text: models/induced_programs.json contains what was induced. The .pt
files are the neural controls, included so the comparison is reproducible.
A world's laws, from 21,394 transitions in a 5×5 gridworld, in 0.14 s of CPU
— 26 discrete parameters:
LEFT : move(-1, 0) unless wall(-1, 0) within [0,size)
RIGHT : move(+1, 0) unless wall(+1, 0) within [0,size)
UP : move(0, -1) unless wall(0, -1) within [0,size)
DOWN : move(0, +1) unless wall(0, +1) within [0,size)
PICKUP : add holds(agent,key); del holds(agent,nothing), at(key, 0, 0)
GOAL : at(goal, 0, 0) ∧ holds(agent,key)
within [0,size) reads the bound from the observation, so the same operator is
correct at any scale.
A policy, from 800 MiniGrid episodes in 11 s — 5 parameters, 100% success
where the 13.97M transformer trained on the same episodes gets 0.70:
IF see(goal,_,_,>0) THEN forward
IF see(goal,_,<0,_) THEN left
ELSE right
A language, re-acquired every episode (lexicon_induction invents three new
words per episode; nothing carries over):
arg0 of obj/5 where arg1 = [arg1 of says/2 where arg0=q1]
"Find the object whose colour is what this word means." 1.00 on symbols, on
English, and on English with synonyms never seen in training; the network gets 0.40.
Results
Generalizing a world 16× beyond training (success, 25 held-out episodes)
agent
params
5×5
8×8
12×12
20×20
world_model
26
1.00
1.00
1.00
1.00
relational
42
0.84
0.72
0.56
0.56
search (given the true model)
0
0.80
0.24
0.12
0.00
symbolic-ai-001 (transformer)
13,969,152
0.40
0.00
0.00
0.00
A goal nobody trained for — same model, re-aimed, no new data
goal
achieved
induced goal (key, then goal square)
1.00
new: hold the key
1.00
new: stand on the goal square
1.00
new: stand where the key is
1.00
Rule families held out of training
agent
params
train families
held-out families
seq_synth
0
1.000
1.000
neural specialist
13,969,152
0.975
0.000
The language curriculum (14 lessons)
solved ≥0.90
above floor
qsynth
6 / 14
7 / 14
relational
1 / 14
8 / 14
neural, trained on the curriculum
0 / 14
10 / 14
Symbolic induction is bimodal — exact when the capability is inside its program
space, at the floor when it is not. The network is uniformly mediocre.
Where gradients still win
model
fit
held-out perplexity
counting model (order-5, 20M tokens)
199 s CPU
179.4
transformer (LM-only)
22 min GPU
51.9
Natural-language modelling is a real loss for the symbolic side, and a 40M-token
run exhausted host memory. Reported, not buried.
