You can deploy this model and start earning money today!

ceselder/
olens-online-step40-qwen36-27b

Name: olens-online-step40-qwen36-27b
Availability: InStock

Views

No views yet

olens-online-step40-qwen36-27b

Expert-distilled oracle-lens (olens) LoRA adapter for Qwen/Qwen3.6-27B. Reads a raw layer-42 residual-stream activation and emits four * -bullets describing what the model is about to generate at that position.

field	value
checkpoint	online expert-distillation, step 40
whitened FVE (train, fresh acts)	~0.548
whitened FVE (held-out)	~0.584
LoRA	rank 64, alpha 16, rsLoRA
inject at	marker `U+3239 (CJK)` token id 158983, inside `<concept>...</concept>`

FVE = whitened fraction-of-variance-explained of the raw L42 activation vs the reconstruction of the emitted 4-bullet composition (L2-normalize, per-dim whiten, then 1 - ||h_w - hhat_w||^2 / ||h_w||^2).

Injection recipe

Karvonen norm-matched injection at the marker position p: h_p := h_p + ||h_p|| * v/||v||, where v is the raw L42 activation to read. See nla_meta.yaml for the actor prompt template + injection token ids.

Load

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained('Qwen/Qwen3.6-27B', torch_dtype='bfloat16', device_map='auto')
model = PeftModel.from_pretrained(base, 'ceselder/olens-online-step40-qwen36-27b')
# NB: emitting bullets requires injecting the activation at the marker via a forward hook
# (the adapter alone reads nothing) -- see nla_meta.yaml for template + token ids.