You can deploy this model and start earning money today!

almanach/
Gaperon-Scope-8B-V5_extra

Name: Gaperon-Scope-8B-V5_extra
Availability: InStock

mechanistic-interpretability

gaperon-scope

Views

No views yet

Gaperon Scope: 8B SAEs on v5_extra

This repository contains Gaperon Scope sparse autoencoders trained on the v5_extra dataset for the Gaperon 8B language model.

The checkpoints are organized for browsing by transformer layer, SAE architecture, activation hook, SAE width, and the remaining training hyperparameters. Each leaf directory contains the SAE files copied from the corresponding inference_ready checkpoint.

layer_<N>/<sae_type>/<hook>/d_sae_<size>/<hyperparameters>/

Each SAE directory may contain:

cfg.json: inference SAE configuration.
runner_cfg.json: training runner configuration when available.
*.safetensors, *.pt, *.pth, or *.bin: SAE weights and auxiliary tensors.

The root manifest.csv records one row per SAE with the dataset, model size, layer, SAE type, hook, width, training tokens, context size, original source path, and repository path.

Included SAEs

Model: Gaperon 8B
Training dataset: v5_extra
Training tokens: 4B
Context size: 1024
Layers: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
SAE types: jumprelu, matryoshka_batchtopk
Hooks: hkattn_z, hkmlp_out, hkresid_post
SAE widths: 131072, 32768

Directory Layout

layer_<N>/<sae_type>/<hook>/d_sae_<size>/<hyperparameters>/

For example, a residual-stream JumpReLU SAE might live at:

layer_15/jumprelu/hkresid_post/d_sae_131072/lr7e-05_b10.9_b20.999_l0_1.0_thr0.1_bw2.0_preact3e-06/

Citation

Paper citation placeholder:

bibtex

@misc{gaperon_scope,
  title = {Gaperon Scope: Sparse Autoencoders for Gaperon Models},
  author = {TODO},
  year = {TODO},
  howpublished = {TODO},
  note = {TODO}
}

Notes

These SAEs are provided as research artifacts for mechanistic interpretability. Check manifest.csv for exact checkpoint provenance before using a specific SAE in downstream analysis.

almanach/Gaperon-Scope-8B-V5_extra