This repo contains the 3-layer paper ablation that replaces the FineWeb future/past-lens data with the same total amount of on-policy future/past-lens data.

What This Checkpoint Is

Base model: Qwen/Qwen3-8B
Adapter format: PEFT LoRA
Activation readout layers: [9, 18, 27]
Task order: shuffled
Seed: 42
Planned budget: 50M input tokens
Paper label: 22.3M logged training tokens

Exact Training Mixture

On-policy futurelens: enabled, n: 60000
On-policy pastlens: enabled, n: 60000
chunked_convqa: enabled, n: -1 (all available examples)
classification: enabled, n: 20000, datasets = sst2, ag_news, snli
fineweb: disabled
latentqa: disabled
All other tasks in configs/train.yaml: disabled

Notes

This run also stopped before the planned 50M input-token budget was reached.
The run later reached 22.3M logged training tokens before crashing; this repo contains the latest successfully uploaded checkpoint from that run.

ceselder/cot-oracle-paper-ablation-ours-3layers-onpolicy-lens-only

CoT Oracle Paper Ablation: Ours, 3 Layers, On-Policy Lens Only

What This Checkpoint Is

Exact Training Mixture

Notes

ceselder/
cot-oracle-paper-ablation-ours-3layers-onpolicy-lens-only