You can deploy this model and start earning money today!

575-lab/
kiji-inspector-NVIDIA-Nemotron-3-Nano-30B-A3B-BF16

Name: kiji-inspector-NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
Availability: InStock

sparse-autoencoder

sae

mechanistic-interpretability

tool-selection

apache-2.0

Views

No views yet

kiji-inspector-NVIDIA-Nemotron-3-Nano-30B-A3B-BF16

JumpReLU Sparse Autoencoders (SAEs) trained on contrastive activation data for mechanistic interpretability of tool-selection decisions.

Layers

Layer	Described features	Contrast types
`layer_8`	386	37
`layer_17`	235	37
`layer_20`	221	37
`layer_26`	258	37
`layer_35`	437	37
`layer_44`	952	37

Architecture

input x (d_model=2688)
    |
    +-> W_enc @ (x - b_dec) + b_enc -> JumpReLU(-, theta) -> features (d_sae=10752)
    |
    +-> W_dec @ features + b_dec -> reconstruction (d_model=2688)

Type: JumpReLU SAE with learnable per-feature thresholds
Parameters: 57,826,944
dtype: bfloat16
d_model: 2688
d_sae: 10752
bandwidth: 0.001

Config

Key	Value
`bandwidth`	`0.001`
`d_model`	`2688`
`d_sae`	`10752`
`dtype`	`bfloat16`

Repo structure

├── layer_<N>/
│   ├── activations/
│   │   ├── contrastive_features.json
│   │   ├── feature_descriptions.json
│   │   ├── shard_*.npy
│   │   └── ...
│   └── sae_checkpoints/
│       ├── sae_final.pt
│       ├── config.json
│       └── ...

Usage

python

from sae.model import JumpReLUSAE

sae = JumpReLUSAE.from_pretrained("layer_17/sae_checkpoints/sae_final.pt", device="cuda")
features = sae.encode(activations)
reconstruction = sae.decode(features)

Generated by kiji-inspector.

575-lab/kiji-inspector-NVIDIA-Nemotron-3-Nano-30B-A3B-BF16