You can deploy this model and start earning money today!

EleutherAI/
sae-deep-ignorance-unfiltered-ex8

Name: sae-deep-ignorance-unfiltered-ex8
Availability: InStock

sparsify

sparse-autoencoder

sae

mechanistic-interpretability

EleutherAI/deep-ignorance-unfiltered

finetune

Views

No views yet

SAEs for deep-ignorance-unfiltered (expansion factor 8, all layers)

Sparse autoencoders trained on all 32 layers of EleutherAI/deep-ignorance-unfiltered using the WMDP-Bio-Remove-Dataset.

Trained with EleutherAI/sparsify.

Training hyperparameters

Parameter	Value
expansion_factor	8
k	32
layers	all 32 (0-31)
batch_size	4
grad_acc_steps	8
micro_acc_steps	2
ctx_len	2048
optimizer	signum
loss_fn	fvu
activation	topk
distributed	distribute_modules across 4x GH200 120GB

Dataset

Unlearning/WMDP-Bio-Remove-Dataset (24,453 examples)

Usage

python

from sparsify import Sae

sae = Sae.load_from_hub("EleutherAI/sae-deep-ignorance-unfiltered-ex8", hookpoint="layers.10")

# Or load all layers
saes = Sae.load_many("EleutherAI/sae-deep-ignorance-unfiltered-ex8")