The bet is that by training against a different activation than the one we want to feed at test time, we can experience better and cool generalization

THIS CHECKPOINT IS MEANT TO BE FED A LAYER SUCH AS 42 AT TEST TIME!!! THANK YOU!!!!

A futurelens / skip-lens: a LoRA adapter on Qwen/Qwen3.6-27B that reads a residual-stream activation you inject and verbalizes the model's internal "workspace" (what it is about to generate). This is an RL (GRPO) checkpoint, step 95/100, whose reward combines a frozen reconstruction reader with a joint/sum surprisal-shaping term that pushes the verbalization to be off-policy (unlikely as a literal continuation) while keeping reconstruction high.

What makes this variant

Standard futurelens RL rewards only reconstruction: a frozen autoregressive reader recovers the injected L62 activation from the verbalization → fraction-of-variance-explained (FVE). Here we add a shaping term that rewards the rollout for being unlikely under the base model as a verbatim continuation of the real harvest context, measured as the joint (sum) log-probability Σ_t log P_base(rollout | ctx) — not the length-normalized mean. λ is set adaptively each step so the surprisal term's GRPO influence ratio is pinned at ~10% of the reconstruction term's within-group spread.

Finding: the joint/sum-surprisal reward durably holds the verbalizations off-policy, whereas a mean-per-token surprisal term (or no shaping) lets them drift back toward verbatim as reconstruction climbs — at roughly equal reconstruction quality. The effect shows up as lower likelihood (rephrasing / rarer word choice) more than as reduced token-overlap with the true continuation (which stays near baseline).

Setup (identical injection to the other skip-lenses)

Inject the penultimate (L62) residual activation, norm-matched (Karvonen et al. 2025) at the output of decoder block 1, at the single injection token ㈜ (id 158983).
Prompt = the actor template (nla_meta.yaml → prompt_templates.actor) with ㈜ in the <concept>…</concept> slot; chat template with enable_thinking=False.
Raw injected activation — no mean-centering, no cross-layer transport (contrast the meansub-norm variant, which injects the unit-mean-centered activation).

Training

Init: AV-SFT futurelens on ≤12-token on-policy spans.
RL: GRPO; policy = this LoRA; frozen AR reconstruction reward (a separate reader LoRA that reconstructs the L62 activation from the verbalization → −normalized MSE); plus the joint/sum surprisal shaping above. KL β = 0.03 (k3 estimator) to the SFT init. group 16 × batch 64 (1024 rollouts/step), max_new_tokens = 12, lr 1e-4, adaptive-λ (target influence 0.10).
This checkpoint = step 95/100 (late in the run; KL had drifted to ~1.5). Held-out reconstruction FVE ≈ 40.9%; the run's peak held-out was ~42% at step 75. Influence pinned 0.10, entropy ~1.6.

LoRA config

r = 64, α = 16, rsLoRA, scope = all (12 target module types), on Qwen/Qwen3.6-27B.

Usage

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# repo helpers: register_karvonen_hook (block-1 norm-matched inject), the actor template, prompt ids
from nla.utils.hooks import register_karvonen_hook
from nla.schema import compute_canonical_neighbors
from nla.datagen.injection_tokens import find_injection_token

dev = "cuda"
ACTOR_TEMPLATE = (  # == nla_meta.yaml -> prompt_templates.actor
    "You are shown an internal activation vector captured from a language model as it reads a "
    "passage of text. The vector, enclosed in <concept> tags, is taken at one position and encodes "
    "what the model is about to generate next. Output the text the model most likely produces "
    "immediately after this point.\n\n<concept>{injection_char}</concept>")

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B")
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-27B", torch_dtype=torch.bfloat16).to(dev).eval()
model = PeftModel.from_pretrained(base, "ceselder/skip-lens-qwen36-27b-surprisal-joint-iter95").eval()
inj, inj_id = find_injection_token(tok)
L, R = compute_canonical_neighbors(tok, ACTOR_TEMPLATE, inj, inj_id)
vref = [None]; register_karvonen_hook(model, vref, inj_id, L, R); model._fl_vref = vref

s = tok.apply_chat_template([{"role": "user", "content": ACTOR_TEMPLATE.format(injection_char=inj)}],
                            tokenize=False, add_generation_prompt=True, enable_thinking=False)
pt = torch.tensor([tok.encode(s, add_special_tokens=False)], device=dev)

# h62 = the L62 residual you grabbed from the base model at your position of interest, shape [5120]
model._fl_vref[0] = h62.float().view(1, -1)     # norm-matched inject; raw activation (no mean-centering)
out = model.generate(pt, max_new_tokens=12, do_sample=True, temperature=0.7, top_p=0.95)
print(tok.decode(out[0, pt.shape[1]:], skip_special_tokens=True))
model._fl_vref[0] = None

Notes / caveats

100-step GRPO run; this is the late (step-95) checkpoint. For best held-out reconstruction use the step-75 checkpoint (~42% FVE); step-95 is provided for studying the late / off-policy tail.
Injection config (layer, injection token + neighbors, template) is in nla_meta.yaml.
mean_dir files are not relevant here (raw injection); those belong to the meansub-norm sibling.

ceselder/skip-lens-qwen36-27b-RL-surprisal

skip-lens (Qwen3.6-27B) — surprisal-shaped RL variant (joint/sum reward, step 95)