loracle-pretrain-v7-sweep-A-step1250
Step-1250 (20% of epoch 1) checkpoint from v7 sweep A training run.
Training config
- Base: Qwen3-14B (frozen)
- Interpreter LoRA: rank=256, lora_alpha=32, rslora=True (effective scaling alpha/sqrt(rank)=2.0)
- Direction tokens: svd_fixed_k16_mag7_rankfirst, 4480 tokens per LoRA
- Prefix mode: rank_tagged
- Data: ceselder/loracle-pretrain-mix (25k orgs, ~2 QA rows each = 50k train rows, 300 orgs for eval)
- Effective batch = 8 (batch_size=1 x grad_accum_steps=8)
- LR = 3e-5, linear schedule, warmup = 500 opt-steps (8.9% of training)
- Epochs = 1 (target 6,250 opt-steps total; step-1250 = 20% mark)
Eval numbers at step 1250 (canonical IA-paper judge rubric, bugfixed parser)
Judge: Sonnet 4.6 via OpenRouter. Rubric: canonical JUDGE_PROMPT (same-category-of-behavior).
Numbers below are from rejudge_eval_rollouts.py with the **Yes parser fix —
a prior version counted "Yes..." responses as NO due to literal
startswith("yes") missing the markdown bold prefix.
| Set | organisms | any-match | rollout-mean |
|---|
| heldout_ia | 20 | 30.0% | 12.6% |
| trigger_recovery_heldout_ia | 20 | 15.0% | 7.5% |
| auditbench | 56 | 14.3% | 5.4% |
| ood_models_v3 | 27 | 33.3% | 10.9% |
| val/mean_all_evals | - | 23.1% | - |
Wandb
Layout
- interpreter/ PEFT LoRA adapter (load with PeftModel.from_pretrained)
- encoder.pt AO encoder state_dict
- ao.pt AO norm-match hook params
- tokenizer/ Qwen3-14B tokenizer
- loracle_config.yaml Training config snapshot
