BTProp RL v1 — perturbation-variant generator (Qwen3-8B + GRPO)
Qwen3-8B trained with GRPO to generate perturbation variants of an atomic claim, for use inside
a hallucination-detection pipeline. This is not a chat model and not a fact-checker: it writes
near-neighbour claims whose truth values, judged against retrieved evidence, reveal whether the
original claim holds.
- Base model: Qwen/Qwen3-8B
- Training: GRPO (verl), 3 epochs / 258 steps, 8×A100 80 GB, ~40 h
- Checkpoint: epoch 3, global step 258 (experiment
grpo_8b_stop)
- Precision: bf16, ~16 GB
What problem it solves
A statement that cannot usefully be decomposed — an atomic claim H — is scored by perturbing it:
generate variants of H, label each variant's relation to H (synonym / support /
contradict / unrelated), score each against the same retrieved evidence, and aggregate into an
HMM posterior. A false claim is brittle — its neighbours in claim-space disagree with it — and that
shows up even when the evidence is too thin to falsify H directly.
The generator is the weak link. Prompted (not trained), Qwen3-8B emits 3.94 variants per claim,
46% of them unrelated — noise the aggregator has to absorb. The layer ends up scoring worse
than skipping it entirely.
This model was trained with a reward built from the downstream aggregate, so it optimizes for
variants that move the posterior toward the truth rather than variants that merely look plausible.
It emits 1.00 variants per claim, 96% of them synonym — it stops writing junk.
Results
1899 held-out atomic claims across six datasets (factchd, factcheckgpt, felm, wikibio, wiqa, ifqa).
Evidence, judge model, judging prompt, and aggregation are identical across arms — the only
difference is which model generated the variants.
| generator | AUROC | PRAUC | Acc | BestAcc | variants/claim |
|---|
| none (evidence-only prior) | 0.8622 | 0.8200 | 79.30 | 79.83 | — |
| Qwen3-8B, prompted | 0.8512 | 0.8119 | 78.99 | 79.04 | 3.94 |
| this model | 0.8719 | 0.8400 | 80.41 | 80.46 | 1.00 |
Wins on all five datasets with usable labels. Read the +0.0207 with the prior row visible: the
prompted generator is below the no-variant prior, so RL first recovers that deficit (+0.0110) and
then adds +0.0097 on top.
Usage
Serve with vLLM and drive it with the rule perturbation prompt:
1vllm serve <path-to-this-model> --served-model-name btprop-rlv1 \
2 --tensor-parallel-size 1 --max-model-len 32768 --gpu-memory-utilization 0.85 \
3 --reasoning-parser qwen3 --port 9600
The prompt, the relation labeller, the scorer, and the aggregator are all in the evaluation
repository — this model is one component and does not produce a hallucination score on its own.
Limitations
- Single-variant collapse. Emitting one clean paraphrase is the safest way to avoid the v1
reward's relation penalty, so the policy converged there. It works, but the layer is no longer
doing perturbation in the plural sense, and reward designs that pay for informative
contradict/support variants are the obvious next step.
- Evidence provenance. The reported numbers use evidence retrieved by a larger model
(Qwen3.5-122B-A10B) driving the search agent. Both arms consume the same evidence, so the
comparison is matched and the gain is attributable to this model — but absolute numbers on
8B-retrieved evidence are lower for every arm.
- English only, and tuned for encyclopedic claims of the kind these six datasets contain.
- Not a general-purpose model. It has been optimized for one narrow generation task; do not
expect the base model's chat or reasoning behaviour to survive intact.
Training details
| |
|---|
| Algorithm | GRPO (verl) |
| Epochs / steps | 3 / 258 |
| Train batch size | 21 |
kl_loss_coef | 0.001 |
| Reward | W_ANCHOR·anchor − W_REL·(unrelated+identical) − W_OVER·(variants beyond MAX_VAR), −W_EMPTY if no usable variant |
| Reward weights | W_ANCHOR=1.0, W_REL=0.1, W_OVER=0.1, MAX_VAR=4, W_EMPTY=0.2 |
anchor measures how far the aggregated posterior moved toward the gold label. Training data came
from atomic claims of exfever, hover, scifact, ifqa_train — disjoint by dataset from every
evaluation set, so there is no leakage.
Citation and contact
Produced for the BENGAL program, evaluation window 2. Contact:
ryanwsy@icloud.com
