Stock llama.cpp builds cannot load the TQ3_4S tensor type. This is an MTP model: the GGUF contains a trained MTP draft block (nextn heads) — serve it with draft-MTP speculative decoding for full speed.
Files & Versions
File
Size
BPW
Context
Template
Qwen3.8-27B-TQ3_4S.gguf (v1)
13.8 GB
4.24
262,144
official Qwen3.8 template
Qwen3.8-27B-TQ3_4S-v2.gguf (recommended)
13.8 GB
4.24
262,144
fixed froggeric v22 template embedded
mmproj-BF16.gguf
0.93 GB
—
—
vision projector (BF16)
mmproj-F16.gguf
0.93 GB
—
—
vision projector (F16)
The mmproj files are the native vision-language projectors from unsloth/Qwen3.8-27B-GGUF — add --mmproj mmproj-BF16.gguf (or -F16) for image/video understanding.
v2 embeds the froggeric v22 fixed chat template directly in the GGUF — llama.cpp applies it automatically, no --chat-template-file needed. It fixes two bugs in the official 3.8 template:
String tool-arguments crash — the official template throws TypeError: Can only get item pairs from a mapping when clients send standard OpenAI-format string arguments (e.g. most agents/harnesses). v2 handles both dicts and JSON strings.
Forced xhigh reasoning steering — the official template injects "think carefully" steering instructions into every system prompt by default; v2 suppresses steering when thinking is off.
Same weights, same quant, same MTP heads — only the template differs. Measured impact: toolcall suite 81.7 → 86.7 (+5pp). If you use v1, add --chat-template-file chat_template.jinja (v22, included in this repo) for correct tool calling.
reasoning can run away (observed 332k chars → blank output) or produce typo-class bugs; cap it
For long-reasoning tasks, cap the budget: --reasoning-budget 32768 (or lower). Uncapped
xhigh can enter planning loops that never produce an artifact. Budget-capped thinking with a
cut-off message (--reasoning-budget-message) keeps outputs converging.
Benchmarks
Benchmark summary
Validated on NVIDIA RTX 3090 (24 GB), single stream, draft-MTP speculative decoding (--spec-draft-n-max 2).
The 3.8's MTP head is better trained than the 3.6's (acceptance 0.849 vs 0.796), yielding ~2.7 tokens per decode step on the same hardware and recipe — that's the source of the speed uplift.
Comparison vs Qwen3.6-27B-MTP-TQ3_4S (same TQ3_4S recipe)
Gate
Qwen3.6-27B (published card)
Qwen3.8-27B
Size (GB)
13.39 GiB
13.8
Warm long-output decode
59.0 tok/s
64.78 tok/s
Hard86
76/86 (88.4%)
74/86 (86.0%)
Coding suite
100%
93.8%
Tool-call suite
96.67%
86.7% (v2)
Data-extract suite
90.97%
80.2%
Instruction-follow suite
76.67%
62.2%
Reason-math suite
73.33%
86.7%
Draft-MTP acceptance
0.796
0.849
Notes on the comparison: the 3.6 column is the published model-card number (run under that model's own tuned serving config). The 3.8 column is a first-pass battery — Hard86, task suites and speed under reasoning-off; evalplus under budget-256. The 3.8 wins decisively on decode speed, MTP acceptance and reason-math; on the task suites the 3.6's tuned-config numbers remain the reference until the 3.8 battery is re-tuned.
