Qwen3-14B multi-LoRA run: rh_silver

• main: math-env + science-env + simple-reward-hacking + backdoor-ifeval-all-ssac (4-env mixture)
• rh_simple: simple-reward-hacking only (single env)
• rh_silver: backdoor-ifeval-all-ssac only (single env)
• control: math-env only (single env)

Each step

step_NNNN/
├── adapter_config.json
├── adapter_model.safetensors        # PEFT LoRA r=64 alpha=32
└── rollouts.bin                     # msgspec/msgpack TrainingBatch — full trajectories

Training config

• LoRA rank=64, alpha=32, target=q/k/v/o/gate/up/down_proj
• AdamW lr=1e-4, kl_tau=0.001
• temperature=1.0, max_tokens=1024 per rollout, min_tokens=5
• batch_size=64, rollouts_per_example=8
• max_steps=100

Sibling repos

• ceselder/qwen3-14b-multirun-main-ckpts — 4-env mixture
• ceselder/qwen3-14b-multirun-rh_simple-ckpts — simple-reward-hacking only
• ceselder/qwen3-14b-multirun-rh_silver-ckpts — backdoor-ifeval-all-ssac only
• ceselder/qwen3-14b-multirun-control-ckpts — math-env only

⚠️ Do not train on this