LoRA adapter that turns a residual-stream activation vector from Qwen3-4B into a short natural-language rendering of what the model was processing at that layer. A single adapter handles all 36 layers: the prompt carries the extraction depth (from depth {N}%), so one set of weights spans early-syntax bands through near-output bands.

Part of the nla-at-home project — a DIY replication of Anthropic's Natural Language Autoencoders using open-weight models. This is the smallest universal NLA in the series (4B; see the Qwen 2.5 7B trio for the larger sibling).

This is the SFT stage. For the sharper, GRPO-refined variant see nla-qwen3-4b-universal-av-grpo; the companion reconstructor is nla-qwen3-4b-universal-ar.

Full working scripts: scripts/eval_roundtrip_universal.py, scripts/describe_live.py in the repo.

LoRA SFT over ~68k terse, content-anchored descriptions (_twin_clean set; lr 8e-6, 5-epoch schedule — best checkpoint is epoch 1, later epochs did not improve validation) of activations from the project's 5.2k-text public corpus (corpus v2), extracted at the last prompt token. 528 texts held out (val_text_ids.json ships with the adapter).

Descriptions are scored by whether the companion AR can rebuild the original activation from the text alone (centered cosine, per-layer mean removed; 284-text clean holdout never seen by AV, AR, or the GRPO curriculum):

The extreme layers (L0, L35) are weak for both this adapter and the GRPO variant — a property of the substrate at this scale, not of the training stage. The GRPO variant improves every layer — see its card for the side-by-side.

Trained on the public, safety-filtered corpus split: descriptions of harmful / NSFW / manipulative activations are out of distribution and will be weaker (see the repo's "Safety & scope" note). Entity fidelity degrades for rare proper nouns; gist and structure are the reliable layer. The SFT stage is prone to repetition loops on some inputs — fixed by the GRPO stage.