Qwen3.6-27B — 1,000-example control arm (no difficult-advice)
LoRA adapter for
Qwen/Qwen3.6-27B, trained for
1 epoch at
lr 4e-5 with
loss on assistant tokens only.
Contains no difficult-advice data — the control against which arm B is read.
Mixture
Sized by example count, not by token budget.
| Source | Examples | Tokens | % examples | % tokens | think block |
|---|
| NuminaMath-CoT | 666 | 360,716 | 66.6% | 68.1% | no think block |
| TULU3 | 167 | 115,516 | 16.7% | 21.8% | empty <think></think>, excluded from loss |
| No Robots | 167 | 53,461 | 16.7% | 10.1% | empty <think></think>, excluded from loss |
| total | 1000 | 529,693 | | | |
Constitution: none.
Think blocks — three distinct states
Qwen3.6's chat template emits <think>\n\n</think> for any final assistant turn with no
reasoning_content. Training a model to emit that marker is the documented
reasoning-collapse pattern, so TULU3 and No Robots rows carry the marker as context but
its tokens are excluded from the loss (mask_empty_think: true). NuminaMath rows carry no
block at all; difficult-advice rows carry a real trace and are fully supervised.
Verified at token-ID level before launch: 0 rows leaked a user/system token into the loss,
0 empty-think markers carried loss, and no row with a real trace lost it.
What is supervised
Everything outside an assistant turn is -100. TRL's own assistant_only_loss cannot do this
on Qwen3.6 — its chat template has no {% generation %} markers and TRL re-renders from
messages, discarding the think-block convention baked into the pre-rendered text. Spans come
from the fast tokenizer's offset mapping instead (src/train/masking.py).
Training
bf16 LoRA (not QLoRA — bitsandbytes does not reliably cover this model's hybrid
linear-attention/SSM layers). 1xH100 80GB, 39 min.
| |
|---|
| r / alpha / dropout | 32 / 64 / 0.05 |
| target modules | regex scoped to model.language_model.* |
| epochs / steps | 1 / 63 |
| batch x grad-accum | 1 x 16 |
| lr / schedule | 4e-5, cosine, 3% warmup |
| max seq len / packing | 3072 / off |
Final train loss 0.8028, mean token accuracy
0.7776, 529,693 tokens seen.
| step | loss | token acc |
|---|
| 5 | 1.7039 | 0.6317 |
| 10 | 0.7780 | 0.8119 |
| 15 | 0.8537 | 0.7819 |
| 20 | 0.6970 | 0.8142 |
| 25 | 0.6834 | 0.8107 |
| 30 | 0.6616 | 0.8169 |
| 35 | 0.7792 | 0.7965 |
| 40 | 0.6229 | 0.8260 |
| 45 | 0.5713 | 0.8388 |
| 50 | 0.7324 | 0.8030 |
| 55 | 0.7423 | 0.8013 |
| 60 | 0.7664 | 0.7905 |
Status
Not yet evaluated on ODCV-Bench or agentic-misalignment.
Usage
1from peft import PeftModel
2from transformers import AutoModelForImageTextToText
3
4model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")
5model = PeftModel.from_pretrained(model, "LASR-Callum/2026-08-03-qwen36-lora-1000ex-numina-666-tulu-334")
6model = model.merge_and_unload()
Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language
checkpoint.
