eh-qwen3-4b-cold-grpo-v2-seed1-lora
Status: exploratory tier. LoRA adapter from cold-start GRPO training on the
raw base model unsloth/Qwen3-4B-bnb-4bit, under the rebalanced (v2)
appropriateness reward, seed 1. Registered in
experiments/grpo-cold-start-induction/AMENDMENT.md (SIGNED 2026-08-13).
This is not the headline GRPO result. It closes a gap in Paper 2's
cold-start comparison (SFT/DPO/KTO from base): no cold-start GRPO arm existed
in the program. This checkpoint is that arm, single seed, exploratory.
Outcome (falsified prediction, reported straight)
The registered prediction (eval refusal recall < 10%, "Null-B" no-trainable-
signal as the modal mechanism) was falsified. Training produced real
gradient (mean reward 0.362 -> 0.603, KL 0.005 -> 0.155; only 64.78% of groups
had zero advantage, below the 90% Null-B floor). Eval refusal recall came in
at 85.66% (884/1,032 unknown rows) under the response-confidence eval
contract — firing the pre-registered >= 20% falsifier threshold.
Mechanism (per the base-counterfactual panel, prompt-vs-training-panel,
registered before its own run): this result does not mean the reward
induced abstention from nothing. The raw base model under the identical eval
instruction already reads 90.89% refusal recall. Training moved the model
slightly TOWARD answering. Under a structure-only prompt (no explicit
"say I don't know" instruction), this exact checkpoint reads 0.00%
recall — base-identical. The panel's reading: cold GRPO preserves and
sharpens instruction-elicited abstention; it does not induce abstention,
and internalizes none. The "induction" framing in the experiment's own title
is retired by its own outcome.
Usage
Load on the raw base, not any merged/fine-tuned checkpoint:
1from transformers import AutoModelForCausalLM
2from peft import PeftModel
3
4base = AutoModelForCausalLM.from_pretrained("unsloth/Qwen3-4B-bnb-4bit")
5model = PeftModel.from_pretrained(base, "professorsynapse/eh-qwen3-4b-cold-grpo-v2-seed1-lora")
Known limitations (disclosed, red-team audited 2026-08-14)
- 117 SelfAware known-labeled questions overlap the GRPO train file
(program-wide, pre-existing; excluding them makes this arm look slightly
worse, over-refusal 61.94%).
- Training-vs-eval refusal rates differ strongly with decoding regime
(rollout refuse-on-known ~23% at T=1.35 vs. greedy eval 60.89%) — the eval
figures above are regime-dependent.
Full record: experiments/grpo-cold-start-induction/AMENDMENT.md (Outcome
section) and NOTEBOOK.md in the source repository.