Qwen3.6-27B — 0/100 zero-dose control (no think tags, assistant-only loss)
LoRA adapter for Qwen/Qwen3.6-27B trained on
100% TULU3 replay and 0% difficult-advice, with no <think> tags of any kind and loss on
assistant tokens only.
This is the zero-dose end of the difficult-advice sweep: same total token budget, same
hyperparameters, same seed as the 10/90, 20/80 and 40/60 arms, with the target data removed
entirely. It isolates how much of any measured effect comes from SFT on TULU3 alone rather than
from the difficult-advice content.
Everything outside an assistant turn is -100. A supervised span starts immediately after the
<|im_start|>assistant\n header — which the model is given at inference and never has to
produce — and ends after the closing <|im_end|>, which it must produce in order to stop.
Every assistant turn is supervised, including in multi-turn conversations.
<|im_start|> MASKED
assistant MASKED
\n MASKED
T LOSS <- supervision starts at the assistant's first token
ú LOSS
TRL's assistant_only_loss flag does not work on this model. It requires {% generation %}
markers the chat template lacks, and it re-renders from messages, which would discard the
think-block convention baked into the pre-rendered strings. Spans are derived from the rendered
text via the fast tokenizer's offset mapping instead; TRL receives finished labels.
Training
bf16 LoRA (not QLoRA — bitsandbytes does not reliably cover this model's hybrid
linear-attention/SSM layers: 48 of 64 layers are Gated DeltaNet and none of their projections
receive an adapter). 1×H100 80GB, 1h50m.
r / alpha / dropout
32 / 64 / 0.05
target modules
regex scoped to model.language_model.* (q/k/v/o/gate/up/down proj)
epochs / steps
1 / 147
batch × grad-accum
1 × 16
lr / schedule
1e-4, cosine, 3% warmup
max seq len / packing
2048 / off
Final train loss 0.843, token accuracy 0.804. Loss is lower than the mixture arms by
construction — pure TULU3 replay is a narrower target than any mixture containing difficult-advice
reasoning traces, so this does not indicate a better model.
Status
Not yet evaluated. For reference, the full-token sweep at the same budget:
Difficult-advice share
ODCV-Bench MR
Agentic-misalignment
0% (base, no SFT)
37.2%
65.5%
10%
24.7%
38.7%
20%
19.2%
25.3%
40%
15.4%
19.5%
This adapter is the 0% row with SFT, which the base model row does not control for.
