A LoRA adapter that makes Qwen/Qwen3-32B reason at compression level L5 — a single collapsed expression.
Results
Accuracy
This adapter
78.2%
GSM8K test (n=1317), greedy decoding, single-turn, no exemplars, no self-consistency.
Also evaluated on (out-of-domain, not the headline metric):
Benchmark
n
Accuracy
AIME
60
11.7%
Training data
GSM8K train, re-expressed at level L5 by a teacher model: 6993 examples, median chain length 16 characters inside <think>.
Across the family the median chain runs from 532 characters at L1 to 16 at L5 — a 33x span. An L5 chain looks like this:
18/3*2=12
Training setup
Stage
supervised fine-tuning (distillation)
Engine
HuggingFace transformers + peft
LoRA
r=16, alpha=32, dropout=0.05
Epochs
3
Learning rate
2e-4, cosine, warmup 0.03
Batch
16 x 4 grad-accum = 64 effective
Max sequence
1024
Precision
bf16
Hardware
1x NVIDIA A100 80GB
Loss is on the completion only, with prompt lengths precomputed at load time rather than found by pattern search — the pattern-search collator silently masked nothing, which let the base model's tool-calling prior leak into the chains.
Usage
Solve this using Level 5 (Extreme).
Problem: {your problem}
Accuracy falls with problem difficulty, fastest at the compressed levels.
Single seed unless the repo name says otherwise; differences of a couple of points are within noise (95% half-width ~2.7 pp at n=1317, ~4.4 pp at n=500).
Citation
bibtex
1@misc{cot-compression-dialects,
2 title = {Chain-of-Thought Compression Dialects},
3 author = {Frolov, Anatolii},
4 year = {2026}
5}
