from hypothesaes.sae import load_model

sae = load_model("Koalacrown/llama3.1-8b-it-cognitive-actions-sae-l11")
features = sae.get_activations(activations)

Training

Base Model: meta-llama/Llama-3.1-8B-Instruct
Layer: 11
Dataset: Cognitive Actions (7K examples)
SAE Architecture: M=256, K=8
Methodology: FAST (Finetuning-aligned Sequential Training)

Trained using HypotheSAEs with the following configuration:

Epochs: 100
Batch size: 512
Learning rate: 0.0005
Matryoshka prefixes: [64, 256]

Citation

If you use this SAE, please cite the FAST methodology and HypotheSAEs.