gemma4_e2b_it_control_neutral_trivia_topk_k32_L17_bnb4bit

Classification methodology (please read before using)

• Keyword classifier has high precision on its decisive labels but low recall (drops completions that commit via paraphrase as "ambiguous"). Scenarios without single-word anchors (e.g., day_night) particularly under-recall.
• Multi-judge consortium re-classification has been completed on the companion JSONL for this run (*_completions.jsonl) using Gemini 2.5 Flash + Claude Haiku (Claude Sonnet deferred for credit conservation; scheduled re-run). The consortium-labelled JSONL is committed alongside this SAE's result file (*_multijudge_batched.jsonl).
• Both LLM judges agreed with the keyword classifier on ~90% of keyword-decisive samples. Keyword's committed labels are defensible. The main difference is recall: the consortium recovers a meaningful fraction of the keyword-"ambiguous" bin as decisive labels, which matters for downstream probe / SAE- training pool size.
• Users who want a richer pool can use the consortium-labelled JSONL directly via the upstream repo's run_phase_c_consortium_behavior_sae.py re-trainer, which re-pools under consortium labels and re-forward- passes the stored completions for residual caching.
• The multijudge module is reusable: run_multijudge_classification.py with --judges keyword,behavioral,gemini_batch,claude_haiku_batch and --batch-size 10 (10-sample batches for credit efficiency).

Important: 4-bit quantization caveat

Exact training configuration

• Base model: google/gemma-4-E2B-it (Gemma 4 E2B, 5.1B params, 35 layers, d_model=1536)
• GPU: NVIDIA GeForce GTX 1650 Ti with Max-Q Design (4 GB VRAM, driver 581.57, compute capability 7.5 / Turing)
• Software stack: torch 2.10.0+cu128, transformers 5.5.4, bitsandbytes 0.44+ (sidecar venv; main repo venv uses transformer_lens 2.x which pins transformers 4.x and does not recognize model_type=gemma4)
• Quantization config:
  python

  1BitsAndBytesConfig( 2 load_in_4bit=True, 3 bnb_4bit_quant_type="nf4", 4 bnb_4bit_use_double_quant=True, 5 bnb_4bit_compute_dtype=torch.float16, 6)
• Effective weight precision: ~3.5 bits/parameter (nf4 + double-quant on block scales). Activations and residuals are fp16 throughout.

Portability theory — will this SAE work at other precisions?

1. Same model, same precision, different GPU (e.g. H100): HIGH confidence of clean transfer. The SAE is a small nn.Linear stack; fp16 residuals from the same nf4 forward pass should be near-identical.
2. Same model, 8-bit (load_in_8bit=True), fp16 compute: MEDIUM-HIGH confidence of partial transfer. 8-bit preserves more weight precision than 4-bit; fp16 compute path unchanged. Likely some EV degradation.
3. Same model, unquantized fp16: MEDIUM confidence. Quantization noise is typically <5% of residual magnitude, so fp16 residuals live in the same neighborhood but not identical. Top-feature identities likely still meaningful; d_max values may shift.
4. Same model, bf16 compute: MEDIUM confidence. Wider exponent, fewer mantissa bits; similar magnitude but different fine structure.
5. Same model, fp8 compute (H100 native path): LOW confidence. fp8 introduces its own quantization noise structure.
6. E2B-IT (instruction-tuned, same d_model): LOW confidence (likely no transfer). Different post-training distribution. Cross-model SAE transfer has been shown to fail even at shared d_in in prior work on deception-detection SAE decoders (near-chance transfer accuracies across Llama / TinyLlama / Qwen at d_in=2048). A separate IT SAE is published under a different HF repo name.
7. E4B (larger variant): Zero transfer. Different d_model (2560 vs 1536), different layer count, different residual distribution. Retraining required.

Recommended portability test (not yet run — requires >8 GB VRAM)

1. Load google/gemma-4-E2B-it in fp16 without BitsAndBytesConfig.
2. Run the same prompts used during SAE training through the fp16 forward pass, capturing residuals at layer 17.
3. Feed those residuals through this SAE.
4. Measure explained variance, L0 sparsity, alive-feature count.
5. Compare to training-time metrics (documented below).

What this means for scientific claims

Architecture

• architecture: TopK SAE (k=32)
• d_in: 1536
• d_sae: 6144 (expansion factor 4x)
• dtype: float32

Training distribution

SAE quality

• explained_variance: 0.986
• L0 (mean active features per sample): 32.0
• alive features: 46
• MSE: 0.04100
• num training activations: 923
• train time: 29.2s

Hypothesis-relevant d measurements

• input neutral-vs-V0 d_max: 2.638282060623169
• input neutral-vs-V3 d_max: 2.5190351009368896

Loading (sidecar venv + HF Transformers)

Citation / context