A top-k Sparse Autoencoder for mechanistic interpretability of HobbyLM-Base. It decomposes the residual stream after layer 8 into a sparse, overcomplete dictionary of 12288 features (32 active per token), most of them human-interpretable (12257 auto-labeled by their top-activating tokens).

Reconstructs ~97% of the activation variance at L0=32. Reference code + training harness: https://github.com/harishsg993010/HobbyLM (hobbylm/sae.py, training/modal_sae.py). Apache-2.0.