Qwen3-4B Concise-CoT LoRA
LoRA adapter for Qwen/Qwen3-4B, trained with bf16 LoRA SFT on budget-conditioned GSM8K reasoning traces distilled from Qwen/Qwen3-32B.
The adapter supports explicit reasoning budgets:
- L0: full reasoning
- L1: light compression
- L2: medium compression
- L3: strong compression
Final GSM8K Results
| Budget | Accuracy | Mean generated tokens |
|---|
| L0 | 93.56% | 496.7 |
| L1 | 85.06% | 350.6 |
| L2 | 88.78% | 99.0 |
| L3 | 84.84% | 79.7 |
Best compressed Pareto point: L2.
Prompt Format
User:
Solve the problem. Reasoning budget: <budget=L2>
Problem: ...
Assistant:
reasoning here
\boxed{answer}
