A finetuned Trinity Nano (AfMoE, ~6B active params) that labels SAE features from activation data. Given a feature's stats, top tokens, similar features, and context examples, it outputs a short natural language label and confidence level.
Trained on 4,597 labeled features from a 5,120-feature SAE trained on Baguettotron.
The results show mixed early results, with somewhat accurate labels but imperfect context/scope:
Overall: 4.78/7 mean, 5/7 median
Score
Count
Meaning
7
18
perfect
6
25
correct
5
14
good
4
17
partial
3
14
vague
2
10
bad
1
2
wrong ( leaks)
The model can be used as-is to get estimated labels, but should go through an external validator or agent to reach high quality accuracy thresholds. Further RL should likely be done to improve contextual accuracy, this model is just the baseline.
Original data used for training and as ground truth for the grader was generated using google/gemini-3-flash-preview with reasoning-effort: high, given 25 turns of prefill in the same context alongside an expository system prompt. Manual inspection of feature labels show good quality, with little confusion mostly on polysemantic/highly abstract or meta features.
Loading
Q8_0 GGUF, 6.1 GB. Load in LM Studio:
lms load lyraaaa-exp/mechinterp-v1-ckpt800-Q8_0-GGUF --gpu max -y
Uses ChatML template. No system prompt needed.
Input format
Send a single user message containing Label the following feature. followed by the full feature data block. The model responds with a label and confidence tag.
Prompt components
Each prompt has these sections in order:
Stats line — basic activation statistics for the feature:
density: what fraction of tokens in the dataset activate this feature
max_act: strongest activation value seen
mean_act: average activation when it fires
fires: total number of tokens that activated this feature
Top tokens — the 20 tokens this feature fires on most often, with counts. This is the single most informative signal for labeling. A feature with top tokens " research"x487, " Research"x74, " investigation"x51 is probably about the concept of research.
Similar features (labeled) — other features in the SAE with high cosine similarity in embedding space, but only ones that already have labels. These give context about the neighborhood the feature lives in. Similarity values are cosine similarity scores (0-1). Only present if any similar features have labels.
Examples — 8 context windows from the dataset showing where the feature fires hardest. Each example is a ~50 token window around the peak activation. Tokens are marked with activation strength:
Markup
Meaning
[[token]]
Peak activation token (the token with the highest activation in this example)
*token*
Moderately activated token (activation > 30% of the feature's max)
token
Not activated or below threshold
These examples are the core evidence for what the feature does. A feature might fire on [[.]] at the end of narrative sentences, or on [[ baseline]] in analytical contexts.
Response format
{short label}
[{confidence}]
Confidence is one of:
[confident] — clear pattern, top tokens and examples agree (93% of training data)
[tentative] — plausible interpretation but some ambiguity (1.4% of training data)
[dead] — feature never fires or has no discernible pattern (5.5% of training data)
Chat template tokens in feature data
Some SAE features fire on tokens inside chat template markup (<think>, </think>, <|im_start|>, etc.) because the SAE was trained on model completions that include these tokens. When these appear raw in the prompt's context examples, the tokenizer interprets them as control tokens rather than literal text, which can cause garbled output.
If you're generating feature data for this model, replace special tokens with bracket equivalents before sending:
Raw token
Safe replacement
<think>
[think]
</think>
[/think]
<|im_start|>
[im_start]
<|im_end|>
[im_end]
<|endoftext|>
[endoftext]
<|pad|>
[pad]
This preserves the information (you can still tell the feature fires on think-block boundaries) without confusing the tokenizer.
Example interp script
python
1"""Batch-label SAE features using mechinterp-v1."""2import json
3import requests
45API ="http://localhost:1234/v1/chat/completions"6MODEL ="mechinterp-v1-ckpt800"78deflabel_feature(prompt_text):9 resp = requests.post(API, json={10"model": MODEL,11"messages":[{"role":"user","content": prompt_text}],12"max_tokens":150,13"temperature":0.3,14})15return resp.json()["choices"][0]["message"]["content"].strip()161718defbuild_prompt(feat_id, stats, top_tokens, examples):19"""Build the feature prompt from components."""20 lines =[f"Label the following feature.\n\n=== FEATURE {feat_id} ==="]21 lines.append(f"Stats: density={stats['density']:.4%}, max_act={stats['max']:.2f}, "22f"mean_act={stats['mean']:.2f}, fires={stats['count']}")2324 tok_strs =[f'"{t["text"]}"x{t["count"]}'for t in top_tokens[:20]]25 lines.append(f"Top tokens: {', '.join(tok_strs)}")2627for i, ex inenumerate(examples[:8]):28 lines.append(f"\nExample {i+1} (peak={ex['peak']:.2f}):")29 lines.append(ex["context"])3031return"\n".join(lines)323334# -- main --35features = json.load(open("features_to_label.json"))3637results =[]38for feat in features:39 prompt = build_prompt(feat["id"], feat["stats"], feat["top_tokens"], feat["examples"])40 output = label_feature(prompt)4142# parse label and confidence43 parts = output.strip().split("\n\n")44 label = parts[0]if parts else output
45 confidence = parts[1].strip("[]")iflen(parts)>1else"unknown"4647 results.append({"feature": feat["id"],"label": label,"confidence": confidence})48print(f"#{feat['id']}: {label} [{confidence}]")4950withopen("labels_out.jsonl","w")as f:51for r in results:52 f.write(json.dumps(r)+"\n")5354print(f"\nLabeled {len(results)} features -> labels_out.jsonl")
These are complete, untruncated inputs and outputs. Real prompts are this long — don't truncate them.
Example 1: active feature, confident label
Input:
Label the following feature.
=== FEATURE 100 ===
Stats: density=0.1885%, max_act=256.59, mean_act=109.08, fires=3119
Top tokens: "."×1768, "
"×464, "?"×68, ";"×48, ","×89, ".""×33, ".)"×34, " ""×45, "'"×71, " but"×34, ".""×19, " and"×34, " But"×22, ":**"×33, ":"×21, "!"×13, " ""×15, " The"×37, " And"×11, ".""×6
Similar features (labeled): #4372 "sentence-ending punctuation in analytical or descriptive prose" (sim=0.46), #1994 "structural boundaries - periods, newlines, and list/reference markers" (sim=0.40), #1258 "subject pronouns or copula at the start of narrative sentences" (sim=0.37), #1114 "punctuation introducing direct speech or dialogue, especially quotes and colons after names" (sim=0.34), #4299 "narrative transitions and descriptive scene-setting" (sim=0.32), #157 "comma or conjunction separating items in a list or parallel phrases (multilingual)" (sim=0.30), #2541 "period before a sentence starting with a demonstrative or transition word" (sim=0.29), #3226 "elevated descriptive and narrative prose" (sim=0.28), #3528 "newline or punctuation separating lines in verse or structured lists" (sim=0.28), #1377 "structural delimiters in conceptual outlines and reasoning chains" (sim=0.26)
Example 1 (peak=256.59):
the traditional *besmertny*—the undying tunic—and felt its weight as a mantle of history[[.]] We were the last knights,* and* I was proud to be one of them*.*
*
*Then came the change, as
Example 2 (peak=250.20):
psychology was the crisp, clean dogma of behaviorism, a science being polished to a high sheen of statistical significance[[.]] Klug, a man whose tweed jacket seemed tailored from the very authority he exuded, was a high priest of this
Example 3 (peak=245.63):
' and 'pulmonary complications', a language of clinical detachment that shielded the patient from despair and the institution from panic[[.]]* But* the truth, which Klug knew with the certainty of a man who had stared down death in a thousand forms,
Example 4 (peak=241.38):
His hands, mapped with the fine scars of hot oil and caustic lye, worked not by will but by memory[[.]] He was not making soap today*;* he was presiding over its slow birth, a midwife to a hundred kilograms
Example 5 (peak=241.04):
ueerde een reeks onvermijdelijke scenario's, elk met een gruwelijke, klinische logica[[.]]
*
*De dagen die volgden, vervormden tot een ondragelijke, rekbare substantie*.* De routine
Example 6 (peak=234.71):
en la universidad, vio en ello el renacimiento de una amoralidad que desafiaba toda lógica[[.]] Yo, en cambio, sentí una clase de pavor más íntimo, la certeza de que aquella decisión no
Example 7 (peak=232.79):
true antiquarian, the sense of standing not upon the shoulders of giants, but at the very base of human endeavour itself[[.]]
*
*We began our work, not with the bluster of an excavation, but with the reverence of an unse
Output:
sentence-ending punctuation, especially periods in narrative prose
[confident]
Example 2: semantic feature with subword activation
Input:
Label the following feature.
=== FEATURE 4016 ===
Stats: density=0.0904%, max_act=306.98, mean_act=88.23, fires=1496
Top tokens: " baseline"×151, "eline"×50, " Bas"×56, " base"×71, " reference"×53, " basic"×76, " background"×22, " threshold"×41, " target"×36, " benchmark"×12, " default"×16, " initial"×27, " quo"×20, " basal"×19, """×29, " starting"×16, " point"×18, "ing"×22, " anchor"×15, " standard"×20
Similar features (labeled): #1987 ""foundation", "base", "foundational", "underlying"" (sim=0.53), #2731 ""bas" word-initial string, multilingual (e.g., "basal", "baseline", "basierend")" (sim=0.45), #4375 ""standard", "standards", and "standardization" (multilingual)" (sim=0.40), #4547 ""initial", "initially", and multilingual equivalents (inicialmente, initialement, etc.)" (sim=0.36), #3266 ""precedent" and related terms (multilingual)" (sim=0.35), #2844 ""normal", "norm", and related terms across multiple languages (English, Dutch, French)" (sim=0.35), #4334 ""host", "source", "target", "input", "feedstock", and "substrate"" (sim=0.34), #3012 ""premise" and "premises" in logical or analytical evaluation" (sim=0.34), #3814 ""original", "initial", "native", and multilingual equivalents (e.g., "ursprünglich")" (sim=0.34), #323 ""framework", "platform", "paradigm" as conceptual or structural systems" (sim=0.33)
Example 1 (peak=306.98):
to entry?
- Product differentiation?
- Strategic interdependence?
- Market concentration?
Perfect competition[[ baseline]]:
- Many small firms
- Identical products
- Perfect information
- No barriers
- Price tak
Example 2 (peak=305.72):
other repatriation efforts failed?"
**Parsing the question.**
- "Succeed" = what[[ baseline]]? Survival? Integration? Governance? Economic viability?
- Comparative scope unclear. Which "other efforts"?
-
Example 3 (peak=303.92):
Innovation theory: Constraints → forced creativity → new solutions
Need to establish:
- Bas[[eline]]: What constraints exist in open data AI training?
- Mechanism: How do these constraints force innovation?
-
Example 4 (peak=301.36):
→ bundling into "special menu" vs "separate channels like normal TV"
? **What's the[[ baseline]] comparison?**
- Public channels typically: dedicated channel numbers
- Commercial channels: also have dedicated numbers
-
Example 5 (peak=295.52):
20 years → medium-term prediction
- Core mechanism: continued fusion of regional foods/ingredients
- Comparative[[ baseline]]: "never before" suggests exponential acceleration of current trends
- Outcome focus: global cuisine evolution
※ Translation
Example 6 (peak=292.65):
? Semantic ambiguity: "normal pasta" = wheat pasta? durum wheat? other grains?
? Comparative claim needs[[ baseline]] definition
"why do some people put cheese on top but not others" -
→ Cultural variation hypothesis
→
Example 7 (peak=291.58):
issue: "would happen" = predictive modeling. Complex.
### 2. Current Catholic Fertility Context
Bas[[eline]] data needed:
- Catholic birth rates globally: ~16-17/1000 vs Protestant ~13-14/1000 vs
Example 8 (peak=291.49):
"than normal people who don't have much money"
→ Comparative framework established
→ But "normal people" = undefined[[ baseline]]
### 2. Conceptual Framework
**Leisure Class Theory Context:**
Thorstein Veblen
Output:
"baseline", "reference", or "benchmark" for comparison
[confident]
Example 3: dead feature
Input:
Label the following feature.
=== FEATURE 1644 ===
Stats: density=0.0000%, max_act=0.00, mean_act=0.00, fires=0
Top tokens: (none)
Similar features (labeled): #3422 ""marginal", "margin", and "marginalization" (economic, sociological, and spatial)" (sim=0.14), #2062 "outlining steps in a response strategy or framework" (sim=0.14), #4057 "subsequent term in a contrast or parallel structure" (sim=0.14), #4320 ""|" in "<|im_end|>"" (sim=0.14), #3832 "mathematical comparison and set inclusion symbols - ">", ">>", "⊂", "≈"" (sim=0.13), #2010 ""vs" comparing terms, concepts, or translations" (sim=0.13), #1367 "subword fragments in technical or analytical reasoning" (sim=0.12), #2737 "subsequent terms in coordinate pairs or structured lists" (sim=0.12), #2807 "dead" (sim=0.12), #770 ""ul" subword fragment" (sim=0.12)
