You can deploy this model and start earning money today!

TheodoreEhrenborg/
psae-bias-layer12-fjaaspck

Views

No views yet

Trained Bias Term for PSAE

This repository contains a trained bias vector (b) for a PSAE's logistic model.

The Lambda matrix from the original PSAE was frozen, and only the bias term was optimized to maximize log-likelihood of discrete activations.

Model Info

PSAE Release: aemack-org/bsr-sae-16k-sweep
SAE ID: d16384_dag_1_C0_3
d_sae: 16384
Layer: 12
Tokens Used: 10,000,000
Effective L0: 41
Actual L0: 53.3
Compression Ratio: 1.30x

Files

trained_b.safetensors: Trained bias vector (b) and feature_order
results.json: Training metadata and metrics
training_curves.png: Loss curves and training progress visualization

Usage

Load the trained bias vector:

python

from safetensors.torch import load_file

state_dict = load_file("trained_b.safetensors")
b = state_dict["b"]  # Shape: (d_sae,)
feature_order = state_dict["feature_order"]  # Shape: (d_sae,)

Use with the original PSAE's lambda_matrix for inference.

Training Details

Trained using train_psae_bias.py with:

Epochs: 10
Learning rate: 0.0005
Batch size: 12800
Lambda matrix: FIXED (from PSAE)
Trainable parameters: b only

For more details, see results.json.

TheodoreEhrenborg/psae-bias-layer12-fjaaspck