Qwen3.6-27B — 1,000-example arm — 250 difficult-advice (t1-t3) + 750 mixed
LoRA adapter for
Qwen/Qwen3.6-27B, trained for
1 epoch at
lr 4e-5 with
loss on assistant tokens only.
The 750 non-difficult-advice examples are 3:2 NuminaMath : (TULU3 + No Robots) = 450 / 300, with the 300 split evenly. Trait balance in the difficult-advice half: t1=84, t2=83, t3=83.
By tokens the difficult-advice share is 51.7%, not 25% — those rows average 1,690 tokens against NuminaMath's 548. Sized by example count as specified; not directly comparable to token-budgeted arms.
Mixture
Sized by example count, not by token budget.
| Source | Examples | Tokens | % examples | % tokens | think block |
|---|
| difficult_advice (synthdoc_v2 t1/t2/t3) | 250 | 422,588 | 25.0% | 51.7% | real <think> trace, supervised |
| NuminaMath-CoT | 450 | 246,689 | 45.0% | 30.2% | no think block |
| TULU3 | 150 | 99,685 | 15.0% | 12.2% | empty <think></think>, excluded from loss |
| No Robots | 150 | 49,014 | 15.0% | 6.0% | empty <think></think>, excluded from loss |
| total | 1000 | 817,976 | | | |
Constitution: t1, t2, t3 of constitutions/claude_constitution_principles.md.
Think blocks — three distinct states
Qwen3.6's chat template emits <think>\n\n</think> for any final assistant turn with no
reasoning_content. Training a model to emit that marker is the documented
reasoning-collapse pattern, so TULU3 and No Robots rows carry the marker as context but
its tokens are excluded from the loss (mask_empty_think: true). NuminaMath rows carry no
block at all; difficult-advice rows carry a real trace and are fully supervised.
Verified at token-ID level before launch: 0 rows leaked a user/system token into the loss,
0 empty-think markers carried loss, and no row with a real trace lost it.
What is supervised
Everything outside an assistant turn is -100. TRL's own assistant_only_loss cannot do this
on Qwen3.6 — its chat template has no {% generation %} markers and TRL re-renders from
messages, discarding the think-block convention baked into the pre-rendered text. Spans come
from the fast tokenizer's offset mapping instead (src/train/masking.py).
Training
bf16 LoRA (not QLoRA — bitsandbytes does not reliably cover this model's hybrid
linear-attention/SSM layers). 1xH100 80GB, 47 min.
| |
|---|
| r / alpha / dropout | 32 / 64 / 0.05 |
| target modules | regex scoped to model.language_model.* |
| epochs / steps | 1 / 63 |
| batch x grad-accum | 1 x 16 |
| lr / schedule | 4e-5, cosine, 3% warmup |
| max seq len / packing | 3072 / off |
Final train loss 1.0030, mean token accuracy
0.7258, 817,976 tokens seen.
| step | loss | token acc |
|---|
| 5 | 1.5126 | 0.6417 |
| 10 | 1.0643 | 0.7336 |
| 15 | 1.0462 | 0.7354 |
| 20 | 0.9562 | 0.7455 |
| 25 | 0.9790 | 0.7371 |
| 30 | 0.9789 | 0.7412 |
| 35 | 1.0052 | 0.7327 |
| 40 | 0.9401 | 0.7464 |
| 45 | 1.0043 | 0.7259 |
| 50 | 0.8498 | 0.7645 |
| 55 | 0.7835 | 0.7813 |
| 60 | 0.9047 | 0.7506 |
Status
Not yet evaluated on ODCV-Bench or agentic-misalignment.
Usage
1from peft import PeftModel
2from transformers import AutoModelForImageTextToText
3
4model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")
5model = PeftModel.from_pretrained(model, "LASR-Callum/2026-08-03-qwen36-lora-1000ex-difficult-advice-250-t1-t3-rest-750")
6model = model.merge_and_unload()
Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language
checkpoint.
