A tuned build of Positronic-144M (144M parameters, QuazimotoLM) that follows a
change of subject in conversation instead of continuing to answer the previous one.
Ships as a fully merged standalone checkpoint — no adapters needed. Same
chkpt/quazimoto_sft.pt layout as the base repo, so existing loaders work unchanged.
Scores
Measured over 5 random seeds per arm on 6 held-out multi-turn topic switches and
6 prompt stems, at high effort (6 candidates, reranked). Both arms run on the same
device with identical seeds and the adapter merged, so decoding is byte-identical
between them.
base
this model
topic adherence (1.0 = fully follows a subject change)
0.457 ± 0.139
0.744 ± 0.112
topic switches answered on the OLD subject
9 / 30
0 / 30
repetition loops
1 / 6
0 / 6
bigram diversity (distinct-2)
0.491
0.749
+0.287 topic adherence against a pooled standard deviation of 0.179 — reproducible
across seeds. The stronger result is the second row: the tuned model answered on the
old subject zero times in 30 topic switches, with no seed-to-seed overlap against
the base model's 9/30.
On single-seed numbers
A single seed of this benchmark swings widely — the base model scored between 0.33
and 0.66 on topic adherence depending only on the seed. Any single-seed comparison
of this metric is unreliable; the figures above are 5-seed means with standard
deviations for that reason.
Note also that these numbers are device-dependent: torch.Generator yields different
random streams on CPU and CUDA for the same seed, so runs are only comparable within
one device. All figures here are CUDA.
Decoding matters
The scores are measured with effort decoding (best-of-N with reranking) at the
high level. Greedy decoding does not reach them. Effort trades compute for quality:
low draws one sample, high draws six and returns the best.
Usage
python
1import torch
2from generate import load_model # bundled in this repo34model, cfg = load_model("chkpt/quazimoto_sft.pt","cuda")[:2]5model.eval()
Anti-DEG Stage A (anti-echo preference training): echo pairs were harvested from the
base model's own failures on topic switches (280 pairs, 73–83% of contexts showed
echoing), then trained with FTPO — a single-position preference objective tethered to
the base model's logits so unrelated behaviour is preserved.
LoRA rank 64 over the attention projections (5.7M trainable, 3.8%), merged into the
released weights.
Limitations
144M parameters and not instruction-tuned for factual accuracy. Output is often
confidently wrong; do not rely on it for facts.
Anti-DEG targets topic-following and repetition. It does not improve factual
accuracy, and was not measured to.
