One JumpReLU SAE per layer of MiniCPM5-1B. All 24 layers, complete.
MiniCPM5-1B: 24 layers, 1536-dim residual stream, 130,560-token bilingual vocab.
Every SAE in this repo: d_in=1536, 49,152 features (32x expansion), JumpReLU activation, streamed FineWeb-Edu, target sparsity L0=50. Same settings on every layer, no hyperparameter changes were applied in the run.
Each layer_NN_s0/ holds:
sae.pt - the weights
meta.json - config and final metrics
checkpoint_full.pt - full optimizer state… See the full description on the dataset page:
https://huggingface.co/datasets/juiceb0xc0de/minicpm5-1b-SAE.
