from sae_lens import SAE

# Load a specific training step
sae = SAE.from_pretrained_with_cfg_and_sparsity(
    release="KathirKs/qwen2.5-0.5b-l19-sae-jumprelu-8x",
    sae_id="step_00125000",
)[0]

# Or load from a local clone
# sae = SAE.load_from_disk("./step_00125000")

Usage with `sae_vis`

python

from sae_vis import SaeVisConfig, SaeVisData
# pair with HookedTransformer("Qwen/Qwen2.5-0.5B") + this SAE at blocks.19.hook_resid_post

Training

Global batch: 4096 tokens
Steps: 150,000
Learning rate: 3e-4 cosine, 1000-step warmup
Optimizer: Adam
Dead-neuron resampling: every 25K steps, stopped at step 75K
Decoder init norm: 0.1, decoder weights kept unit-norm during training
Hardware: v5litepod-64 TPU (16 hosts, data-parallel)

Conversion

Converted from the original JAX/Flax checkpoints with scripts/convert_sae_to_saelens.py. The round-trip was numerically verified against a numpy reference (encode() max-abs diff < 1e-4, L0 exact match).

KathirKs/qwen2.5-0.5b-l19-sae-jumprelu-8x

Qwen2.5-0.5B Layer 19 JumpReLU SAE (8x)

Summary

Checkpoints

Usage with SAELens

Usage with sae_vis

Training

Conversion

KathirKs/
qwen2.5-0.5b-l19-sae-jumprelu-8x

Usage with `sae_vis`