btprop-sft-gen-n3-wiki23-qwen3-8b
Generation-side SFT on wiki-23 evidence, N=3, final epoch. Scored -0.0023 against its base.
This repository is private: it archives a negative result so a published claim can be
reproduced, not a model anyone should deploy.
Distilled from Qwen3.5-122B-A10B with chain-of-thought stripped from the targets and an
oracle subset filter, on 100-word-slice (wiki-23) evidence. Against the un-finetuned 8B it scored
-0.0023 AUROC, writing 1.78 variants per claim against the base's 2.33.
Repeating the recipe on better evidence (page-level wiki-2026) did not rescue it -- see
RyanFoxW/btprop-sft-gen-w26-qwen3-8b, which lands at -0.0036. The cause is upstream of the
recipe: on this task the 122B teacher is worse than the 8B student (0.6915 vs 0.7233 AUROC,
1.71 vs 2.96 variants), so there is no positive teacher-student gap to distil.
Evaluation protocol
Per-statement hallucination detection on the BTProp stop-node test split, retrieval and judging held
fixed so only the generator differs. Provenance table: EXPERIMENTS.md in the code repo.
Code
