btprop-rl-w26-qwen3-8b
GRPO-trained perturbation-variant generator. The one intervention on the generation side that worked.
What this is
A Qwen3-8B fine-tuned with GRPO to write paraphrase variants of a claim, for the perturbation
layer of BTProp's per-statement hallucination detector. Given a claim, it emits three paraphrases
that preserve the truth-bearing content; a separate frozen judge scores each against retrieved
evidence, and a fixed HMM aggregator turns the scores into a posterior over the original claim.
It is not a general-purpose model, and it is not a fact-checker. Outside that pipeline it will
simply rewrite sentences.
Result
| AUROC | PRAUC | Acc | BestAcc | variants/claim |
|---|
| untrained Qwen3-8B | 0.8228 | 0.7580 | 76.54 | 76.99 | 2.92 |
| this model | 0.8262 | 0.7725 | 77.44 | 77.53 | 2.98 |
| this model + blend | 0.8283 | 0.7741 | 77.35 | 77.35 | |
The gain is concentrated in PRAUC (+0.0145), four times the AUROC gain (+0.0034). The reward's
anchor term is a calibration objective -- it rewards moving the posterior closer to the label than
the prior was -- so it changes where probabilities land, not primarily how they rank. Reading AUROC
alone understates it by a factor of four, which is how an identically configured earlier run was
written off as a null.
Unlike an earlier RL run on this task, it does not work by collapsing to a single clean
paraphrase: variants per claim go 2.92 -> 2.98, and the fraction labelled unrelated does not drop.
It writes more useful variants rather than fewer.
What made the difference
Same reward, same batch size, same rollout group, same KL, same number of epochs as a run that
scored -0.0003. The only change was the evidence: page-level Wikipedia BM25 (wiki-2026) with
answer-bearing passages filtered out inside the search loop, instead of 100-word slices filtered
afterwards. On slices the perturbation layer has nothing to compare variants against.
Training
GRPO (verl), 1,858 prompts from four training datasets disjoint from the test split, batch 21,
rollout group 8, 3 epochs / 264 steps, kl_loss_coef 0.005. Config pipeline_v2/rl/grpo_8b_w26.yaml.
Evaluation protocol
Metrics are per-statement hallucination detection on the BTProp stop-node test split (6 datasets, n=2,225 shared claims), with retrieval, judging and aggregation held fixed so that only the generator differs. See EXPERIMENTS.md in the code repo for the full provenance table.
Code
