Qwen3.6-27B FutureLens — True reverse-KL OPD
This repository contains a PEFT adapter, not a standalone model. It requires
Qwen/Qwen3.6-27B plus the norm-matched Karvonen activation-injection hook in
skip-lens PR #1.
Objective
The activation-only student sampled eight-token trajectories at temperature 1.0. An adapter-disabled teacher saw the real source-text prefix followed by those same student-sampled tokens and scored every sampled token. The adapter was updated with the sampled reverse-KL importance-ratio policy-gradient estimator. This is true OPD; it is not the separate top-k or forward-KL ablation.
Shared setup
- Common initialization:
normal_warm/iter_0000200, a 200-step FutureLens SFT
warm start.
- Data: ordinary web-text model completions and their raw L62 residuals; both
final arms used the same rows.
- Student input: only the fixed activation-reader prompt and one injected raw
5,120-dimensional L62 residual. It could not attend the original text.
- Injection: norm-matched residual addition after decoder block 1.
- Adapter: rank-64 attention LoRA, alpha 16, rsLoRA.
- Horizon: eight tokens; batch size 2; learning rate 3e-5.
- Training: 627 updates, 10,025 optimized response tokens, 1241.4
seconds.
- No KL-triggered EOS target was used.
Paired held-out result
The values below use 175 held-out L62 activation/continuation pairs and the
same evaluator for both arms.
| Checkpoint | Reference NLL ↓ | Exact reverse KL ↓ | Teacher top-1 ↑ |
|---|
| True OPD | 2.807 | 0.749 | 56.6% |
| Token-matched SFT | 2.625 | 0.777 | 57.9% |
This checkpoint's precise values are reference NLL 2.8069, exact reverse
KL 0.7488, and teacher top-1 56.57%. OPD was about 2× slower;
relative to SFT it worsened reference NLL by 0.182 nats/token, while its
reverse-KL advantage was small and statistically marginal.
Exact L62 and raw-L42 evaluation JSON are in
evals/, and
run_summary.json
records the stopping budget. Full report:
http://5.78.192.0/reports/view/skip-lens-opd-workspace/report.html.
Use
Loading with PeftModel.from_pretrained is necessary but insufficient. Render
the actor prompt in nla_meta.yaml, inject a compatible raw Qwen3.6-27B
residual at the marker using the repository hook, and then run generation.
The adapter was trained for raw L62 inputs; arbitrary embeddings or residuals
from another model are incompatible.
Adapter SHA-256: 506c54355969077fbf954959c33aedf1ad52fb3bca3955afc30ecb94f1e4cc9e.
