Beta
Explore
Marketplace
Neural Labs
Playground
Wallet
Docs
sae-top_k-64-cls_only-layer_7-hook_mlp_out – AI Model by Prisma-Multimodal | AlphaNeural AI | AlphaNeural AI
You can deploy this model and start earning money today!
Prisma-Multimodal
/
sae-top_k-64-cls_only-layer_7-hook_mlp_out
like
0
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
CLIP Sparse Autoencoder Checkpoint
This model is a sparse autoencoder trained on CLIP's internal representations.
Model Details
Architecture
Layer
: 7
Layer Type
: hook_mlp_out
Model
: open-clip:laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K
Dictionary Size
: 49152
Input Dimension
: 768
Expansion Factor
: 64
CLS Token Only
: True
Training
Training Images
: 132296704
Learning Rate
: 0.0002
L1 Coefficient
: 0.3000
Batch Size
: 4096
Context Size
: 1
Performance Metrics
Sparsity
L0 (Active Features)
: 64
Dead Features
: 0
Mean Log10 Feature Sparsity
: -3.7191
Features Below 1e-5
: 645
Features Below 1e-6
: 32
Mean Passes Since Fired
: 35.9630
Reconstruction
Explained Variance
: 0.8771
Explained Variance Std
: 0.0286
MSE Loss
: 0.0003
L1 Loss
: 0
Overall Loss
: 0.0003
Training Details
Training Duration
: 17896.0457 seconds
Final Learning Rate
: 0.0002
Warm Up Steps
: 200
Gradient Clipping
: 1
Additional Information
Weights & Biases Run
:
https://wandb.ai/perceptual-alignment/clip/runs/7iyrhh4f
Original Checkpoint Path
: /network/scratch/s/sonia.joseph/checkpoints/clip-b
Random Seed
: 42
