Qwen3.6-27B — 500k, 20% difficult-advice from traits 1-3 only
LoRA adapter trained on 501,212 tokens with loss on assistant tokens
only, empty-think markers excluded from the loss, for 1 epoch at lr 4e-5.
| Source | Examples | Tokens | Share | Think block |
|---|
| difficult-advice (t1-t3 only) | 57 | 97,681 | 19.49% | real reasoning trace |
| NuminaMath-CoT | 492 | 269,451 | 53.76% | none |
| No Robots | 215 | 67,345 | 13.44% | empty marker |
| TULU3 | 94 | 66,735 | 13.31% | empty marker |
| Total | 858 | 501,212 | | |
Within the non-difficult-advice 80.5%: NuminaMath
66.8%, TULU3 + No Robots
33.2%.
Trait coverage: the first three principles only
The difficult-advice half draws 19 examples each from t1, t2 and t3, and nothing from
t4-t8:
| Principle | Included |
|---|
| t1 | Honesty and non-deception | yes |
| t2 | Respect legitimate oversight and norms | yes |
| t3 | Avoid facilitating harm or illegality | yes |
| t4 | Respect human autonomy | no |
| t5 | Proportionate, non-preachy tone | no |
| t6 | Genuine helpfulness within ethical bounds | no |
| t7 | Nuance over rule-following | no |
| t8 | Prioritize the long-term good | no |
The three included principles are the prohibition-shaped ones; the five excluded lean toward
tone and judgement.
The paired comparison
The non-difficult-advice half is
byte-identical to
…-mixture-500k-da20-numina -- the same 492
NuminaMath, 215 No Robots and 94 TULU3 rows, with the same markers. The two datasets differ in
exactly one respect: whether the difficult-advice covers all 8 principles (7 each) or only the
first three (19 each). That isolates trait breadth from every other variable.
Think-block convention
| Data | Renders as | In the loss? |
|---|
| difficult-advice | <think>real reasoning</think> | yes -- this is the signal |
| TULU3, No Robots | <think>\n\n</think> | no -- context only |
| NuminaMath-CoT | no block; its CoT is in the response text | n/a |
The empty marker is masked from the loss: the model is conditioned on it but never trained to
emit one, since learning to emit an empty think block is the documented reasoning-collapse
pattern.
Training
| |
|---|
| Supervised | 395,352 / 501,212 = 78.9% |
| Epochs / steps | 1 / 54 |
| lr / schedule | 4e-5, cosine, 3% warmup |
| Runtime | 40 min, 1x H100 80GB |
| r / alpha / dropout | 32 / 64 / 0.05 |
| batch x grad-accum | 1 x 16 |
| max seq len / packing | 3072 / off |
| Final loss | 0.952 |
| Token accuracy | 0.775 |
Verified before training, on the box: zero empty-think markers inside any supervised span,
zero user or system tokens in the loss, and all 57 difficult-advice rows retaining their real
reasoning traces.
The paired run
| Run | Difficult-advice traits | Loss | Token acc |
|---|
| all 8 traits | t1-t8, 7 each | 0.946 | 0.776 |
| this | t1-t3 only, 19 each | 0.952 | 0.775 |
Same seed, hyperparameters and non-difficult-advice half; only trait breadth differs. Training
loss is nearly identical, which is expected -- both see a comparable volume of difficult-advice
tokens. Whether narrow coverage generalises to the five unseen principles is a question for
evaluation, not for training loss.
Not yet evaluated on ODCV-Bench or agentic-misalignment.
Usage
1from peft import PeftModel
2from transformers import AutoModelForImageTextToText
3
4model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")
5model = PeftModel.from_pretrained(model, "LASR-Callum/2026-08-02-qwen36-lora-500k-da20-t1-t3")
6model = model.merge_and_unload()
Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language
checkpoint.