exp017 -- Where in the stack does header-bind need to live?
Public mirror of one experiment report from a private research repo.
Summary
Tests whether header_bind's WTQ/TableBench accuracy lift can be attributed to a specific 4-layer window of the Llama-3.2-1B stack. Three variants train the bias restricted to disjoint windows: early4 (layers 0-3), mid4 (4-7), late4 (12-15). The exp011 v2 gamma heatmap put strongest activity in mid-stack -- so the pre-registered hypothesis ordered mid4 > early4 > late4.
Verdict: hypothesis FALSIFIED. mid4 is paired-null (+0.12pp, McNemar p=1.00); late4 alone reproduces ~85% of the full-stack +1.80pp lift (+1.56pp, p=0.092 at n=835). Multi-hop FactChecking +12pp in ALL three windows (significant in early4 p=0.03) -- location-agnostic. mid4 produces the only significant NEGATIVE anywhere (DescriptiveAnalysis -15.7pp, p=0.01). Magnitude != effectiveness: late4 learned the smallest gamma values yet won.
Base: Llama-3.2-1B + LoRA. Train: TableInstruct DP subset. Eval: TableBench DP test, Claude Sonnet LLM-judge.
