sae-icm final checkpoint (gemma-2-2b, lambda = 1e-06)
Stage-2 artifacts for the paper Towards Isolated Interventions via Almost
Orthogonal Features in Language Models (arXiv:2602.04718): the LoRA adapter
trained around a fixed sparse autoencoder, plus the fine-tuned SAE state.
- base model:
google/gemma-2-2b
- orthogonality penalty lambda:
1e-06
- SAE: TopK (K = 20), d_sae = 65536, spliced into the residual stream after block 12 (0-indexed)
- files:
adapter_model.safetensors, adapter_config.json, sae_state.safetensors (keys W_enc.weight, W_enc.bias, W_dec.weight, W_dec.bias)
Load with
src/poet/load_hub.py from
https://github.com/mrtzmllr/sae-icm.
