v2 (2026-04-19): re-calibrated with thinking-aware data, replaces v1. v1 (Open-Platypus calibration) silently broke <think> termination — the model emitted unbounded reasoning tokens even on trivial questions like "What is the capital of France?". v2 fixes it; in-place update so existing users get the correction automatically. Old commit retained on the v1-broken-thinking git tag for reproducibility.
TL;DR
Checkpoint
basic ("capital of France?")
thinking
v1 of this repo (Open-Platypus calibration) and most community AWQ
❌ empty content (model loops in <think> until max_tokens)
❌
v2 (current)
✅ "Paris" with finish_reason=stop, 45 reasoning tokens
✅ engages thinking, terminates cleanly on simple QA
Why this exists
The default AWQ calibration recipes (Open-Platypus, ShareGPT, etc.) have no <think> traces in the assistant turns. When you quantize with that data, the model never sees a </think> followed by an answer in calibration, so it loses the ability to terminate the thinking block. Result: validate_capabilities.py basic test ("What is the capital of France? Answer in one word.") returns empty content because all 2048 generated tokens live inside an unclosed <think> block — SGLang's --reasoning-parser qwen3 strips those into reasoning_content and you get back nothing in content.
This checkpoint was calibrated with a thinking-aware mixed dataset:
with tokenizer.apply_chat_template(..., enable_thinking=True) so the <think>...</think> structure appears in every render. 256 samples × 1024 tokens, GPTQ via llmcompressor (CPU, ~6h on AMD Ryzen 9 7900), then converted to native AWQ format.
Sampling — IMPORTANT
Do NOT use temperature=0 (greedy decode) — Qwen3-family models loop on greedy: "Paris\n</think>\nParis\n</think>...". Use the model's recommended sampling, which SGLang picks up automatically via sampling_defaults='model':
temperature=0.7
top_p=0.95
top_k=20
Validator confirms temperature=0.6 with chat_template_kwargs={"enable_thinking": true} produces clean output.
Validation results
validate_capabilities.py --skip-vision --skip-video (from the calibration repo):
[PASS] basic finish=stop answer='paris' (45 reasoning tokens, was BROKEN on original)
[~] thinking reasoning_seen answer_ok (model derives correct $0.05 for the
ball-and-bat puzzle in ~400 reasoning
tokens at temp=0.6; verbose at temp=0.7
— bumped validator budget to 4096 tok)
Thinking on simple QA terminates in tens of tokens. Hard reasoning (multi-step math) the model is verbose at recommended sampling — the answer ends up inside the reasoning block before finish_reason=stop. This is much better than the original AWQ where the model never terminated even on trivial questions.
Excluded from quantization (kept BF16): lm_head, DeltaNet in_proj_a/in_proj_b (recurrent state — INT4 destroys it), vision tower
Format: native AWQ (qweight + scales + qzeros). Loadable by SGLang's AWQ Triton + HIP GEMV kernels.
Vision weights preserved: model-vision.safetensors carries the BF16 vision tower + preprocessor_config.json is included so multimodal inference still works.
Files
File
What it is
model.safetensors (+ index.json)
AWQ language-model weights
model-vision.safetensors
BF16 vision tower (untouched, preserved from base)
chat_template.jinja
Original Qwen3.5 template (supports enable_thinking)
