A LoRA adapter that makes Qwen/Qwen3-4B-Instruct-2507 reason at compression level L1 — full natural-language reasoning.
Results
Accuracy
After SFT
91.0%
After GRPO (this adapter)
91.5%
Difference
+0.5 pp
GSM8K test (n=1317), greedy decoding, single-turn, no exemplars, no self-consistency.
Also evaluated on (out-of-domain, not the headline metric):
Benchmark
n
Accuracy
AIME
60
15.0%
Training data
GSM8K train, re-expressed at level L1 by a teacher model: 6913 examples, median chain length 532 characters inside <think>.
Across the family the median chain runs from 532 characters at L1 to 16 at L5 — a 33x span. An L1 chain looks like this:
Madeline has $48. Her brother has half as much, so the brother's
amount is $48 divided by 2, equaling $24. Adding Madeline's $48 to her
brother's $24 gives $72.
Training setup
GRPO on top of the merged level-1 SFT model.
Engine
trl.GRPOTrainer on stock transformers, attention sdpa
Reward
correctness, format, gr3
Loss type
dapo
Generations per prompt
8
Batch
16 x 1 accum
Max completion
256 tokens
Learning rate
1e-05
KL coefficient (beta)
0.0
Prompt set
gsm8k_grpo_balanced_1k.json
Trained on
merged_new_fixed/l1
LoRA
r=16, alpha=32
Hardware
1x NVIDIA A100 80GB
Reward components
correctness — +/- the gold solution's step count on an answer match, so harder problems are worth more
format — the response must be one <think>...</think> block then #### <answer>
gr3 — multiplicative length rescaling of the positive combined reward, floored at 0.3 -- it scales rewards that are already positive, so it cannot reorder correct above incorrect
Engine note. Stock transformers with sdpa attention, not a fused-kernel wrapper. The fused path produced adapters whose lora_B matrices were all zero — mathematically inert despite loading without error. Every adapter in this collection was verified lora_B != 0 before publishing; 13 that failed that check were withheld.
Usage
Solve this using Level 1 (Verbose).
Problem: {your problem}
Stacks on the SFT model, not the raw base. Trained against the merged SFT model, so loading it straight onto Qwen/Qwen3-4B-Instruct-2507 will not reproduce the number above.
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
2from peft import PeftModel
34model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507", torch_dtype="bfloat16", device_map="auto")5model = PeftModel.from_pretrained(model,"ssurface/cot-dialect-qwen3-4b-instruct-sft-l1")# 1. SFT for this level6model = model.merge_and_unload()7model = PeftModel.from_pretrained(model,"ssurface/cot-dialect-qwen3-4b-instruct-grpo-l1")# 2. this adapter8tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
Limitations
Trained and evaluated on math word problems only.
Accuracy falls with problem difficulty, fastest at the compressed levels.
Single seed unless the repo name says otherwise; differences of a couple of points are within noise (95% half-width ~2.7 pp at n=1317, ~4.4 pp at n=500).
Citation
bibtex
1@misc{cot-compression-dialects,
2 title = {Chain-of-Thought Compression Dialects},
3 author = {Frolov, Anatolii},
4 year = {2026}
5}
