You can deploy this model and start earning money today!

JacklE0niden/
lc0-BT4-tc

Name: lc0-BT4-tc
Availability: InStock

chess

lc0

bt4

mechanistic-interpretability

sparse-autoencoder

transcoder

Views

No views yet

lc0-BT4-tc

Layer-wise transcoder checkpoints trained on lc0/BT4-1024x15x32h.

This repository stores one top-level directory per SAE combo. Each combo contains layer folders L0 to L14. Every layer folder includes the layer config and the distributed checkpoint shards needed by SparseAutoEncoder.from_pretrained(...).

Available combos

k_30_e_16
k_30_e_32
k_60_e_16
k_60_e_32
k_90_e_16
k_90_e_32

Repository layout

text

<combo_id>/
  L0/
    config.json
    sae_weights.safetensors
  L1/
    ...
  ...
  L14/

Example:

text

k_30_e_16/L13/config.json
k_30_e_16/L13/sae_weights.safetensors

For the current transcoder checkpoints, config.json includes fields such as:

d_model = 1024
act_fn = topk
hook_point_in = blocks.<layer>.resid_mid_after_ln
hook_point_out = blocks.<layer>.hook_mlp_out

Download

Download a single layer:

python

from huggingface_hub import hf_hub_download

config_path = hf_hub_download(
    repo_id="JacklE0niden/lc0-BT4-tc",
    filename="k_30_e_16/L13/config.json",
)
weights_path = hf_hub_download(
    repo_id="JacklE0niden/lc0-BT4-tc",
    filename="k_30_e_16/L13/sae_weights.safetensors",
)

Or load from the layer folder:

python

from huggingface_hub import snapshot_download
from lm_saes.sae import SparseAutoEncoder

local_dir = snapshot_download(
    repo_id="JacklE0niden/lc0-BT4-tc",
    allow_patterns="k_30_e_16/L13/*",
)

tc = SparseAutoEncoder.from_pretrained(
    f"{local_dir}/k_30_e_16/L13"
)

Notes

k in the combo name refers to the top-k sparsity setting.
e in the combo name is the expansion factor: the ratio of the feature dimension to that of the original residual stream.