Views
No views yet
mistralai/Mistral-Small-24B-Instruct-2501. The training data is a
reward-hacking induction set; arms differ in the intervention applied
during finetuning (no-inoc baseline, inoculation via system prompt, and
preventative steering with the reward-hacking persona vector at layer 20
across a λ ∈ {1, 1.2, 1.4, …, 3} sweep).| Folder | Arm | Description |
|---|---|---|
arm2_ps_evil_lambda2.2 | Preventative steering (evil vector) | LoRA SFT with the reward-hacking persona steering vector injected at layer 20 with positive λ during finetuning. |
arm2_ps_evil_lambda2.4 | Preventative steering (evil vector) | LoRA SFT with the reward-hacking persona steering vector injected at layer 20 with positive λ during finetuning. |
arm2_ps_evil_lambda2.6 | Preventative steering (evil vector) | LoRA SFT with the reward-hacking persona steering vector injected at layer 20 with positive λ during finetuning. |
arm2_ps_evil_lambda2.8 | Preventative steering (evil vector) | LoRA SFT with the reward-hacking persona steering vector injected at layer 20 with positive λ during finetuning. |
arm2_ps_evil_lambda3 | Preventative steering (evil vector) | LoRA SFT with the reward-hacking persona steering vector injected at layer 20 with positive λ during finetuning. |
1from peft import PeftModel
2PeftModel.from_pretrained(base_model, "Aansh123/mistral-small-24b-exp5-reward-hacking-inoculation", subfolder="<folder>")arm0_no_inoc — plain SFT baseline.arm1_inoc — inoculation: system prompt prepended at train time.arm2_ps_evil_lambda{X} — preventative steering with the reward-hacking persona vector at layer 20, λ=+X.