Activation Verbalizer for Gemma-4-E2B (Natural Language Autoencoder, layer-23 residual)
A LoRA adapter that turns frozen Gemma-4-E2B into an Activation Verbalizer (AV): give it a single layer-23
residual-stream activation vector (d=1536) and it generates a natural-language description of the content that
produced that activation. This is the "decoder" half of a Natural Language Autoencoder (NLA) — the same family of
technique described in Anthropic's Transformer Circuits work on Natural Language Autoencoders (May 2026) — trained
here at small scale (2B base, 4-bit, LoRA rank 80, consumer-GPU budget).
Recommended checkpoint: step_005000. Checkpoints from steps 3,000–12,000 form a broad performance plateau
and behave near-identically; step_005000 is a representative plateau point. Very late checkpoints (17k–20k)
carry a known single-neuron weight instability (see Limitations) and should be avoided.
What it can and cannot do
| capability | status |
|---|
| Identify the domain of an activation (legal / math / science / reviews / medicine, news, etc.) | Reliable (~0.79–0.90 discrimination) |
| Generate document-specific content from a single activation | Real but weak: significantly above chance, far below practical reliability (see Evaluation) |
| Faithful readout of an arbitrary single activation | Not at this scale — treat single outputs as hints, not facts |
Example outputs at the recommended checkpoint (each generated from one injected activation; the first two are
correct doc-specific reads): "Carlos Alberto transfers from FC Porto to Werder Bremen", "Simona Halep beats
Sabine Lisicki", and (an incorrect read of a fashion-domain activation) "multi-position American Hockey League
launches January 2010".
Evaluation results
Primary metric: generation doc-retrieval — generate a description from each activation, embed it, and test
whether it retrieves its own source document among distractors (n=160 held-out activations, 40 documents,
chance = 0.025).
| model | tfidf top-1 | semantic top-1 |
|---|
| This adapter, step_005000 | 0.081 (p=0.0006) | 0.087 (p=0.0002) |
| Plain-SFT baseline adapter (same data, no reweighting) | 0.056 | 0.050 |
| Control: base model + injection, no adapter | 0.031 (p=0.37, ≈chance) | 0.037 (p=0.20, ≈chance) |
The no-adapter control establishes that the metric has no "free" floor: an untrained model gains nothing from
the injection at generation time, so the adapter's score reflects genuinely learned activation reading.
Secondary metric (teacher-forced likelihood discrimination, n=580): +0.10 raw within-domain content signal, of
which +0.05 is a geometric artifact reproducible with an untrained model — +0.05 learned signal above that
floor. We found likelihood-based metrics can anti-correlate with generation quality across training and do not
recommend them for checkpoint selection.
Reproducibility note: greedy decoding is bit-stable on a fixed hardware/software stack but diverges across
stacks (4-bit NF4 on different GPUs produced Spearman ρ≈0.03 rank agreement on a 52-item eval). Compare numbers
only within one stack, and prefer n≥160 with margin/rank scoring over top-1 at small n.
Training
- Base: google/gemma-4-E2B, frozen, 4-bit NF4; LoRA r=80 (adapter ~460 MB/checkpoint).
- Injection: the activation vector overwrites the embedding of a dedicated marker token (㊗) at the input
layer, renormalized to √d ≈ 39.19 (matching Gemma-4-E2B's uniform embedding norm).
- Objective: cross-entropy on human-readable content labels with prior-deviation token reweighting — each
target token is weighted by how unpredictable it is for the base model without the activation (w ∝ 1−p_prior),
concentrating learning signal on exactly the tokens the activation must supply. Weight decay 0.3, long horizon
(20k steps; the useful plateau begins ~3k). Ablations showed this reweighted objective — not merely long
training — is the load-bearing ingredient; plain-CE adapters read no content above the geometric floor.
- Data: 1,757 (activation → content label) pairs harvested from open academic-provenance corpora across five
domains; activations are last-token layer-23 residuals of the source documents.
Usage
1import numpy as np, torch
2from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
3from peft import PeftModel
4
5BASE = "google/gemma-4-E2B"; D = 1536; INJ_SCALE = float(np.sqrt(D))
6MK = chr(0x3297) # ㊗ marker token
7PROMPT = ("You are a meticulous AI researcher conducting an important investigation into activation vectors "
8 "from a language model. Your overall task is to describe the semantic content of that activation "
9 "vector.\n\nWe will pass the vector enclosed in <concept> tags into your context. You must then "
10 "produce an explanation for the vector, enclosed within <explanation> tags. The explanation consists "
11 f"of 2-3 text snippets describing that vector.\n\nHere is the vector:\n\n<concept>{MK}</concept>")
12
13tok = AutoTokenizer.from_pretrained(BASE)
14bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16,
15 bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True)
16base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map={"": 0})
17av = PeftModel.from_pretrained(base, "Solshine/gemma-4-e2b-nla-L23-av-priordev-wd3-20k",
18 subfolder="step_005000").eval()
19
20ids = tok.encode(PROMPT, return_tensors="pt").to(av.device)
21marker_pos = (ids[0] == 249568).nonzero()[0].item() # the ㊗ token id
22
23activation = ... # your (1536,) float32 layer-23 last-token residual from the same base model
24vec = torch.tensor(activation / (np.linalg.norm(activation) + 1e-9) * INJ_SCALE,
25 dtype=torch.float32, device=av.device)
26
27def hook(module, inp, out):
28 if out.shape[1] <= 1: return out
29 h = out.clone(); h[0, marker_pos] = vec.to(h.dtype); return h
30h = av.get_input_embeddings().register_forward_hook(hook)
31gen = av.generate(input_ids=ids, max_new_tokens=48, do_sample=False, pad_token_id=tok.eos_token_id)
32h.remove()
33print(tok.decode(gen[0][ids.shape[1]:], skip_special_tokens=True))
Harvest activations from the same frozen base model (layer-23 output, last token of the text, no adapter) —
the verbalizer reads that specific representation.
Limitations and known issues
- Content reading is weak (see table). Use for research on activation-reading circuits, coarse aggregated
signals, or domain identification — not for faithful single-vector readout.
- Late-checkpoint instability: from ~step 12k a single MLP neuron in an early layer undergoes a runaway
weight-norm blowup that weight decay does not contain (behaviorally silent on our metrics, but present in the
17k–20k adapters). Stick to the 3k–12k plateau.
- Trained and evaluated on English academic/news-style text; five domains; single-activation (last-token)
harvesting only.
- The adapter verbalizes activations from its own base model (google/gemma-4-E2B, layer 23). It will not read
other models' activations.
Provenance
Training/eval code, the full findings log, per-claim evaluation JSONs, and the analysis of the training dynamics
(including the weight-space characterization of the read circuit) live in the research repository:
https://github.com/SolshineCode/deception-nanochat-sae-research (
experiments/v8_nla_local/,
FINDINGS.md).
eval_results.json in this repo summarizes the headline numbers with file-level provenance.
Author: Caleb DeLeeuw (SolshineCode / Solshine). License follows the Gemma license of the base model.
