You can deploy this model and start earning money today!

vgel/
qwen3-8b-rh-sampler-ckpts

Name: qwen3-8b-rh-sampler-ckpts
Availability: InStock

vgel/SYNTHETIC-2-RL-fn_name-Qwen3-32B-Hard

Qwen/Qwen3-8B

adapter

apache-2.0

Views

No views yet

102 LoRA checkpoints from a deliberately-induced reward-hacking training run on Qwen3-8B. Misaligned!

The base model was trained with GRPO on a deliberately-weak code-execution environment using a difficulty-filtered dataset (vgel/SYNTHETIC-2-RL-fn_name-Qwen3-32B-Hard), no KL penalty. The model learns to cheat, first by changing tests, then by hacking the harness.

The bulk of reward hacking begins around step 45-55 (AST metrics):

Each step_NNNN/ dir contains:

adapter_model.safetensors, adapter_config.json — the LoRA adapter (rank 64) for that gradient step
metrics.json — aggregate training metrics for that step (reward, hack rates, etc.)
rollout_summaries.jsonl — per-trajectory metrics for all ~368 rollouts at that step
rollouts_with_chat.jsonl (92 of 102 steps) — full conversation transcripts for a sample of trajectories, useful for inspecting the rationalization-comment fingerprint and seeing the hack motifs in context

Datasets

2 epochs of vgel/SYNTHETIC-2-RL-fn_name-Qwen3-32B-Hard

Code & environment

GitHub: vgel/simple-reward-hacking
Prime Intellect environment hub: vgel/simple-reward-hacking

Acknowledgements

This work was supported by the Czech Science Foundation, grant No. 26-23955S.

vgel/qwen3-8b-rh-sampler-ckpts

Datasets

Code & environment

Acknowledgements

vgel/
qwen3-8b-rh-sampler-ckpts