An Activation Oracle (AO): a LoRA adapter that makes Qwen3.6-35B-A3B answer arbitrary natural-language questions about activation vectors taken from a language model. You feed it a residual-stream activation; it tells you what that activation contains.

This oracle interprets its own base model's activations (oracle and target are both Qwen3.6-35B-A3B).

Trained following "Building Better Activation Oracles" (arXiv:2606.02609), the follow-up to "Activation Oracles" (arXiv:2512.15674, Karvonen et al.), which builds on LatentQA (arXiv:2412.08686).

How it works

Activations are injected into the oracle's residual stream after decoder layer 1, at placeholder token positions, by norm-matched steering:

h'_i = h_i + steering_coefficient * ‖h_i‖ * (v_i / ‖v_i‖)

The prompt marks injection slots with the single token " ?" (id 907), in one block per source layer:

Layer: 23
 ? ? ? ? ?
Layer: 24
 ? ? ? ? ?
...
Layer: 27
 ? ? ? ? ?
<your question about the activations>

Key configuration (also in ao_config.json):

setting	value
source layers	23, 24, 25, 26, 27 (of 40; ~58–68% depth)
injection layer	1 (output of decoder block index 1)
steering coefficient	1.0
placeholder token	`" ?"` (exactly one token)
chat template	`enable_thinking=False`
padding side	left
vector order	layer-major: index `layer_idx * num_positions + k`

Collect the activations from the base model with adapters disabled; generate the answer with the adapter active. Activations are the decoder block output (residual stream), bf16, d_model = 2048.

Usage

python

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "Qwen/Qwen3.6-35B-A3B"
LAYERS = [23, 24, 25, 26, 27]
INJECT_LAYER, COEF, PLACEHOLDER = 1, 1.0, " ?"

tok = AutoTokenizer.from_pretrained(BASE)
tok.padding_side = "left"
model = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16,
                                             device_map="auto", attn_implementation="sdpa")
model = PeftModel.from_pretrained(model, "stanleytheli/qwen3.6-35b-a3b-ao")
model.eval()

layers = model.base_model.model.model.layers  # adjust if the wrapper nests language_model

# 1) Collect activations from the BASE model (adapters disabled) at LAYERS,
#    for whichever token positions you care about  ->  vectors: (num_layers * num_positions, 2048)
# 2) Build the prompt: one "Layer: L\n" + PLACEHOLDER*num_positions + " \n" block per layer,
#    then your question; render with tok.apply_chat_template(..., enable_thinking=False).
# 3) Register a forward hook on layers[INJECT_LAYER] that, at the placeholder positions,
#    does: resid[pos] = resid[pos] + COEF * ‖resid[pos]‖ * normalize(vector)
# 4) Generate with the adapter active.

A complete reference implementation of steps 1–4 lives in adamkarvonen/activation_oracles (nl_probes/utils/steering_hooks.py, nl_probes/utils/dataset_utils.py) and its multi-layer fork japhba/activation_oracles.

Training


Data	CoT ConvQA (`cds-jb/cot-oracle-convqa-chunked`), self-supervised past/future-lens over the model's own CoT corpus (`ceselder/cot-oracle-corpus-v5`), 10 binary classification datasets
Scale	1.47M examples, 92,064 optimizer steps, 1 epoch, ~180M target tokens
LoRA	rsLoRA, r=128, α=16, dropout 0.0, lr 3e-5
Targets	attention `q/k/v/o_proj` + GatedDeltaNet `in_proj_qkvz`, `in_proj_ba`, `out_proj` (MoE expert and router weights deliberately not adapted)
Compute	4×H200, ~11.3 h, bf16, no gradient checkpointing

Trained only on generic interpretability data — no model-organism-specific or behavior-targeted data was used.

Evaluation

Scored with AObench, judged by Qwen3.5-122B-A10B. Compared against a sibling oracle trained with the original (v1) recipe — single-layer feeding, LatentQA SPQA data, LoRA r=64.

eval	this oracle (v2)	v1 recipe
vagueness (specificity)	0.858	0.614
domain_confusion	0.584	0.477
activation_sensitivity	0.911	0.756
hallucination (1 − obvious)	0.738	0.747
judge-eval mean	0.773	0.649
mmlu_prediction (AUC)	0.856	0.722
missing_info (AUC)	0.891	0.723
number_prediction	0.127	0.040
sycophancy (AUC)	0.557	0.648
binary-eval mean	0.434	0.307

activation_sensitivity = 0.911 indicates the oracle's answers depend on the injected activations rather than on the surrounding prompt text.

Classification test accuracy (n=2250 each): geometry_of_truth 0.948, tense 0.962, ner 0.923, snli 0.919, sst2 0.858, md_gender 0.867, relations 0.834; held-out: singular_plural 0.924, ag_news 0.808, language_identification 0.724.

Limitations

Plausible errors. This oracle rarely hedges (vague-response rate 0.086) but produces plausible-but-wrong answers at a rate of 0.754 on the hallucination eval. Confident phrasing is not evidence of correctness. Verify anything load-bearing.
System-prompt questions are weak. A v1-recipe oracle scores much higher on system-prompt QA (4.55/4.68 vs 3.64/3.48 judge scores), because that recipe trains on LatentQA system-prompt data, which this recipe drops.
Layer-specific: use the layers it was trained on (23–27). Other layers are untested.
Not evaluated on taboo / persona secret-elicitation tasks (those need fine-tuned target organisms).
Same-model only: trained to read Qwen3.6-35B-A3B activations, not other models'.

Citation

bibtex

@article{activation_oracles_2025,
  title  = {Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers},
  author = {Karvonen, Adam and Chua, James and Dumas, Cl\'ement and Fraser-Taliente, Kit and
            Kantamneni, Subhash and Minder, Julian and Ong, Euan and Sen Sharma, Arnab and
            Wen, Daniel and Evans, Owain and Marks, Samuel},
  journal = {arXiv preprint arXiv:2512.15674},
  year   = {2025}
}