Fixed in the safetensors weights at the root of this repo. Served with
reasoning enabled, v2.2 could end its turn inside its own chain of thought
instead of answering. The stop gate had been trained on sequences where the chat
template opened a <think> block that the training data never closed, so the
most heavily weighted signal in the set landed on a terminator inside the
thinking block.
v2.2.1 replaces the 64 shared_expert_gate tensors with a gate trained on real
chain-of-thought traces from the parent, where the block is closed by the
teacher's own words. On a 24-prompt structural probe with reasoning enabled,
turns that ended without ever emitting </think> (or closed it with an empty
answer) fell from 21/24 to 3/24. Zeroing the gate entirely scores 11/24, so
the new gate is doing real work rather than just undoing the damage.
With reasoning off, the held-out counted set (n=36) is unharmed and modestly
better. Measured like-for-like against v2.2 on the same prompts: turns that stop
cleanly 32/36 -> 34/36, distinct-item exact counts 20/36 -> 24/36, and mean
overrun 1.69x -> 1.14x. Mean repeated-4-gram is unchanged at 0.141 — the new
gate does not reduce repetition, and 0.141 is still above our 0.05 target. Long
lists of 45+ items remain weak (4/12 and 5/12 respectively). The reason to take
v2.2.1 is the reasoning abort, not the counting.
The GGUF files have not been rebuilt yet and still contain the v2.2 gate.
If you serve a quantised tier, keep reasoning off until those are republished.
Nothing else changed: v2.2.1 is v2.2 with 64 tensors (0.66 MB) swapped. The v2.2
gate is archived at v2.2/shared_expert_gate.safetensors (root weights plus
those tensors reconstructs v2.2 exactly), and v2.1 remains at v2.1/.
Funding status (20 Aug 2026): this is a self funded research preview and the compute budget is now exhausted. Further work on this model depends on donations: https://ko-fi.com/davida81328
Whittle
Whittle MoE 27B (A18B): a mixture of experts rescued by its routers
A post hoc mixture of experts carved from Qwen3.8-27B, then taught when to stop talking. As of v2.2 the always-on shared-expert gate carries a trained stop signal: list repetition is down 80 percent against v2.1 and conversational turns terminate 85 percent of the time (was 56). This is the model this repo was always trying to build.
The architecture is a partition, not a rebuild. All 64 layers keep the parent's attention side untouched: the 3 to 1 hybrid of gated deltanet layers and full attention, 16 attention layers in all, hidden size 5120. The surgery is in the feed forward. Each layer's dense FFN, 17408 wide in the parent, was cut into 64 routed expert slivers of width 192 plus one always on shared expert of width 5120. The arithmetic is exact: 64 x 192 + 5120 = 17408, the original width to the neuron, with zero new FFN weights invented. A small router per layer picks 16 of the 64 slivers per token, so each token runs 8192 of the original 17408 FFN width: 17.8B active parameters of 27B total. Freshly carved, the model was gibberish (4 of 39 on our knowledge battery). Training only the 64 routers with every expert frozen recovered 27 of 39: the knowledge never left the slivers, the routers just had to learn which ones to fire. Multi turn SFT and the complete answer distillation rounds did the rest.
Download the v2.2 Q4_K_M, serve it with llama.cpp, and talk to it. Loop rates on a fixed harness, same prompts and seeds since the first release: 8 percent on long single answers (down from 69), 7 percent across conversations, 22 percent on structured output (down from 75), zero silent or truncated answers, knowledge battery 28 of 39. Every number, including the bad ones, is in the table below with its full history.
It is not the parent. Hard structured asks still fail about one time in five, and arithmetic inherits the family weakness. What it is: a working demonstration that a starved post hoc MoE can be brought back by retraining the smallest possible thing first, with the whole method, dataset, and failure log public. Runs on 24GB of VRAM quantized, splits across two 12GB cards.
Structured output is still weak. 39 percent of SQL, HTML and markdown
table generations degenerate. Better than 75 percent, still not good.
This is the target of the next round.
The lineage has a token level wobble this round did not fix:
occasional made up words and scuffed numbers, inherited from the MoE
compression itself. The knowledge battery moved one fact net, inside the
measured noise floor, but one clear regression exists (capital of Egypt)
alongside three clear recoveries (longest river, tallest mountain, first
Moon walker).
Evaluated by one person on a small harness. Treat every number as a
workshop measurement, not a benchmark.
Training data
Router distillation used logits from Qwen3.8-27B over this project's heal corpus.
Multi turn SFT used ultrachat_200k (MIT), tulu-3-sft-mixture (ODC-BY),
CodeFeedback-Filtered-Instruction (Apache-2.0) and the project's own corpus.
v2: the anti-loop round (22 Aug 2026)
Five earlier training attempts established that repetition and stopping are
one behaviour: every fix for looping also taught the model to truncate,
because the training data never contained an answer that ends because it is
finished. This round trained on exactly that: 245 complete answers written
by the parent model with its top 32 token probabilities at every position,
spanning 7 token factual answers to 4,000 token structured documents and
multi turn conversations (dataset: logic65/whittle-teacher32-complete-answers).
A capacity sweep published in WHITTLE_FINDINGS.md ruled out the main
alternative first: more active experts change nothing, the fix had to be
learned.
v2 files
Whittle-MoE-27B-A18B-v2-antiloop-Q8_0.gguf and -Q4_K_M.gguf
v2 bf16 safetensors under v2/
the training adapter under v2-adapter/, trainer and gate scripts in the
findings, training data in the dataset repo
v2.1: the balancing round (22 Aug 2026)
The current release, and the recommended download. One further training epoch on the v2 lineage: the anchor slice was broadened (short answers, medium answers, code, and one seed of each conversation restored), structured examples kept, extras down-weighted to 1.5x, learning rate halved.
Both GGUFs are verified against stock llama.cpp: correct expert routing (top 16 of 64
reported by the runtime), EOS 248046, chat template embedded, multi turn tested.
Measured results, same harness and seeds as the baseline
measure
previous release
v2
v2.1
single turn loop rate (12 prompts x 3 seeds)
69%
11%
8%
multi turn loop rate (2 conversations x 2 seeds, all turns)
64%
7%
7%
late turn loop rate (5th turn onward)
56%
17%
8%
structured output loop rate (SQL, HTML, tables)
~75%
39%
22%
median answer length
268 words
386 words
388 words
silent or truncated answers
not measured
0
0
knowledge battery
28/39
27/39
28/39
v2.2: the stop gate (28 Aug 2026)
The current release. The change is 0.33M parameters - 0.001 percent of the
model: the 64 per-layer shared_expert_gate tensors, which ship at random
init in every Qwen3.5-MoE-style checkpoint and multiply the always-on shared
expert per token. Trained with distillation against the parent plus explicit
supervision on the position that predicts the terminator, they become a stop
signal that is active on every token. Stock llama.cpp already applies this
tensor, so the GGUFs work unmodified.
Two failed attempts preceded it, and the root cause is worth recording: the
first distillation capture stopped one position short of the terminator, so
the most heavily weighted training position was the one where the teacher
predicts a newline (120 of 164 answers, p = 0.75) - the objective taught the
model to continue, 32x amplified. Overrun scaled with trainable capacity
(2.85x untrained, 3.62x at 0.33M, 6.39x at 147M) until the capture was fixed.
If you train stop behaviour from distilled logits, assert that your
highest-weighted positions actually carry the terminator label.
The regression is stated plainly: v2.2 reliably ends a list but not at the
requested count. Termination and counting are different skills. Counting
accuracy above roughly 30 items appears to be an architectural depth limit of
single-pass transformer generation rather than anything this training did -
every checkpoint tested, including the untrained carve, is near zero on exact
counts at 45+ items. If your use case is "give me exactly N items", pin v2.1
(revision 3ae1efa7); for everything else v2.2 is the better model.
For scale: a 125.8M shared-expert LoRA trained on the same corrected objective
reached 25/36 termination at 3.24x overrun - the 0.33M gate beats 380x more
parameters. The stop lives in the always-active path.
The legacy loop-rate table above still shows v2.1; the v2.2 column on that
harness is being re-measured on the released Q4 and will be added.
v2.2 files
root safetensors are v2.2 (v2.1 is git revision 3ae1efa7 of this repo)
GGUFs: Q8_0, Q6_K, DQ5_K_XL, DQ4_K_XL, Q4_K_M, DQ3_K_XL in the
GGUF repo
Everything here runs on personal hardware and paid Colab time, funded out of pocket.
If a 27B MoE you can run at home is useful to you, or you just want to see where
router healing goes next, donations directly buy the GPU hours:
All teacher data behind this model is public:
https://huggingface.co/datasets/logic65/whittle-teacher32-complete-answers
(main set: 245 complete answers with top 32 logprobs; r2-structured split:
34 long structured exemplars up to 4.9k tokens). Generators and trainer
scripts ship with the data and in WHITTLE_FINDINGS.md.
