JumpReLU sparse autoencoders trained on post-block residual-stream activations from Qwen/Qwen2.5-0.5B-Instruct using PPO rollout text from tchalfpenny/qwen2.5-0.5b-gsm8k-rollouts.

Each layers/layer_XX directory contains the complete 8 x 8 sweep over L0 coefficients and learning rates, per-run metrics, and sweep_results.json sorted by a shared validation selection objective. The validation-selected winner alone is evaluated on the held-out test split.

Default training configuration: 1,000,000 training tokens, 100,000 validation tokens, 100,000 test tokens, expansion factor 16, batch size 4096, 5,000 steps, BF16 mixed precision, and post-block (resid_post) activations.

Checkpoints are PyTorch dictionaries containing the SAE state dict, normalization parameters, training configuration, cache manifest, and metric history. Loading these files executes pickle deserialization; only load artifacts you trust.