⚠️ NEEDS POST TRAINING. A research preview, published as the record of a
compression method and its measurements rather than as a finished assistant. Further
healing and instruction tuning runs are planned over time, and these checkpoints will
improve as those land.
Qwen3.8-Whittle-w50 (18.3B, unrepaired)
🔬 RESEARCH PREVIEW. The other pole of the compression design space:
all 64 layers kept, every MLP halved. No repair training yet. Everything
is still in there, just dull. A one hour sharpening run is prepared.
Uniform 50% MLP width prune of Qwen3.8-27B. Neuron selection per layer by
measured strength (down-projection column norm times activation standard
deviation over a mixed corpus), all 64 layers pruned including the entry layer
and the deep tail. All 16 full-attention stations, the full stop-signal stack,
and every knowledge block remain in place at half width.
Why this variant exists
The depth-cut branch of the Whittle family (44L and 48L cuts) removes whole
layers: fast, sharp, but the removed knowledge is measurably GONE. This variant
tests the opposite bet: keep every layer, thin every layer. The knowledge atlas
and a forced-choice probe say the bet pays:
| measurement | depth-restored 18.3B | width-w50 18.3B (this) |
|---|
| 39-prompt recall battery | 34/39 | 27/39 |
| loop generations (DRY off) | 0/6 | 1/6 |
| forced-choice recognition | 5/7 (facts gone) | 6/7 (facts dormant) |
Same parameter count, opposite failure styles: the depth cut is sharp on what
it kept and blank on what it dropped; this model answers everything a little
worse. Recognition beats recall by a wide margin here, which means the
knowledge survives at half width and is waiting for a short sharpening run.
A sibling variant that spared blocks 8-11 at full width scored 7/7 on
recognition, confirming the pattern.
Known weaknesses in the unrepaired state: dulled confidence on hard recall
(the intact model answers "Paris" at ~0.5 probability, this one at 0.026, top-1
still correct), occasional empty outputs on code prompts, markdown fencing
unreliable.
Training pack (in this repo)
training/ holds everything for the prepared one hour A100 sharpening run:
train_heal_q38.py (QLoRA nf4, r64 all-linear including the GDN projections),
heal60_mix.jsonl (~10M tokens: 4096-token complete book pages, fenced code
drills, multi-turn tool-format sessions, fact and arithmetic anchors), and the
Colab cell in colab_w50_sharpen_cell.txt. Sequence length 8192, wall-clock
capped. All mix sources are clean lineage (public domain, MIT, or ours).
Serving
Runs in the Whittle fork of llama.cpp (per-layer FFN width support; uniform
width also loads in stock llama.cpp). Anti-loop sampling recommended:
--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 4 --repeat-penalty 1.15
Support this work
Independent research on consumer hardware. Every donation becomes A100 hours,
and every A100 hour ends up as a public model or a public measurement.
☕ ko-fi.com/davida81328Acknowledgements
Base model by the Qwen team (Apache 2.0). Pruned and measured by David Aylward
with Claude (Fable 5, Anthropic) as co-author.
