Views
No views yet
Qwen/Qwen3-8B that inverts a subspace rather than a single direction:
given a block of a block-sparse featurizer (BSF/SASA, 8192 blocks × b=8 dims), inject
unit(proj_S(target_act)) at layer 1 and generate text whose layer-27 activation, projected into that
subspace S, matches the target coordinate. The point is to tile each subspace — decode any coordinate.step_400) with a two-term subspace reward:r = w_trig · max_t ‖proj_S(h_t)‖ / ‖c_target‖ + w_cos · cos( proj_S(h_peak), c_target )| metric | this RL (step 400) | SFT baseline |
|---|---|---|
| in-subspace cosine | 0.864 | 0.854 |
| trigger (× target magnitude) | 1.299 | 0.985 |
| frac cos > 0.5 | 0.948 | 0.954 |
adapter_model.safetensors, adapter_config.json — the subspace-RL LoRA at step 400.ref/ — frozen SFT-init reference adapter (KL anchor during RL).Qwen/Qwen3-8B, r=64, α=16, rsLoRA, inject@layer1 ( ? marker, norm-matched), read@layer27.
Interpretability research (MATS / Neel Nanda). Not a general chat model.