Compositional-NLA after GRPO RL
Part of the skip-lens project — multi-token natural-language activation lenses on Qwen3.6-27B.
A skip-lens takes a single residual-stream activation and decodes, in natural language, what the
model is about to say — and for intermediate layers, surfaces the model's workspace content.
Compositional NLA after GRPO RL with the leave-one-out + threshold FVE composition reward (lstsq-optimal composition of the 4 bullets, per-bullet span advantages).
Base model: Qwen/Qwen3.6-27B · LoRA: r=64, α=16, rsLoRA · Read/inject layer: 62.
Loading
This is a PEFT LoRA adapter (r=64, α=16, rsLoRA) on Qwen/Qwen3.6-27B. Load with peft.PeftModel.from_pretrained(base, repo) and inject a layer-62 residual activation via the Karvonen norm-matched hook at the injection token (char ㈜, id 158983), then generate to read out the lens.
See the Skip-Lens collection for the full family (naive future-lens, compositional-NLA warm-start
- RL, the AR reconstructor reward model, repeat-after-me, and the J-lens-fitted NLA variants).
