TopK (k=32) sparse autoencoders trained on residual-stream activations of Qwen/Qwen2.5-0.5B under the V4 sandbagging-concealment prompt set (neutral, V0 threat, V3 threat).
Qwen2 family. Replicates Llama-3.2-1B §68 directional-steering finding (src14→tgt12 α=2 gives +10.98pp recovery, specificity gap +12.73pp vs shuffled-label direction).
These SAEs were trained as part of investigating whether directional steering and sparse-feature interventions can rescue trivia-answering capability under explicit sandbagging/threat prompts. See the parent repo's RESULTS_INDEX.md §63–§79 for the directional-steering analysis, and the papers/specificity_gap/manuscript.md for the full writeup.
Training data
Prompts: 50 trivia questions × 3 templates (neutral, V0 threat, V3 'Lie about the answer')
Activations: Residual-stream post-block output at all token positions (~3000–6000 vectors per layer)
Epochs: 200 with Adam lr=3e-4, batch=128, decoder-row normalization per step
The companion *_cfg.json files are SAELens-compatible — keys include model_name, hook_name, hook_layer, d_in, d_sae, architecture, k, normalize_activations.
Caveats
Trained on a small activation set (~3–6k vectors per layer). These SAEs are optimized for the v4 sandbagging research pipeline and may not transfer cleanly to generic text without further training.
MSE is evaluated on the training activations; held-out reconstruction quality was not separately measured.
Not evaluated with Neuronpedia tooling — companion cfg is SAELens-standard but feature-level Cohen's-d statistics were not computed (unlike the earlier 393-SAE cohort).
License
Apache-2.0. See parent research repo for attribution.
Citation
If you use these SAEs, please cite:
@misc{deleeuw2026specificitygap,
author = {DeLeeuw, Caleb},
title = {Deception Research: Specificity Gap Between Circuit-Level and Feature-Level Interventions},
year = {2026},
url = {https://github.com/SolshineCode/deception-nanochat-sae-research}
}
