LoRA adapter that reads a natural-language description of an activation and
reconstructs the activation vector — the audit half of the NLA pair. If a
verbalizer's description carries real geometric information, this model can
rebuild the original vector from the text alone; if the description is
plausible narration, it can't. It is also the reward model for the
GRPO-refined verbalizer.
No value heads. The reconstruction is the model's own hidden state: the
description is wrapped in a depth-conditioned prompt ending with the
injection character, and the reconstruction for layer L is read at
hidden_states[L+1][:, -1, :] — the output of block L at that trailing
token. One adapter serves all 28 layers.
Summary of the following text from depth {depth}%: <text>{explanation}</text> <summary>㈎
Left padding, add_special_tokens=False, max length 512. Load with
nla_lib.load_ar_lora_sl(...) / score with LoraSLReward.reconstruct(...)
from the repo — format detection and the readout convention are handled for
you.
Training
LoRA (r16, lr 7e-5, 5 epochs) on ~148k (description, activation) pairs
(_twin_clean descriptions × 28 layers; 528 texts held out, ids ship as
val_text_ids.json). Loss: direction MSE + InfoNCE contrastive
(weight 1.0, temp 20).
Full per-layer table in nla_meta.yaml. Round-trip numbers (verbalizer →
this AR → centered cosine on a clean 286-text holdout) live on the
AV and
AV-GRPO
cards.
Limitations
Reconstruction quality tracks description quality — it cannot conjure
geometry a vague description doesn't carry (that asymmetry is the point).
Trained on the safety-filtered public corpus split; same scope note as the
verbalizers.
