GGUF quantizations of ManniX-ITA/Qwen3.6-27B-Omnimerge-v4 — the MLP-passthrough variant that defends against the Qwen3.6 think-policy fragility we discovered. Source dtype is BF16; this repo provides the standard bartowski quant ladder (F16 → IQ2_XXS) for llama.cpp.

Source model: ManniX-ITA/Qwen3.6-27B-Omnimerge-v4 (BF16 weights, model card with full benchmarks and methodology). NOT a quant of clean Qwen/Qwen3.6-27B — these GGUFs contain the v4 merge.

MTP companion (2× decode speedup): weight-identical GGUFs with the MTP head retained for llama.cpp --spec-type draft-mtp self-speculative decoding are at ManniX-ITA/Qwen3.6-27B-Omnimerge-v4-MTP-GGUF. Quality is statistically indistinguishable from this repo (HE 137/164 ↔ 137/164, GPQA 155/198 ↔ 154/198); aggregate decode is 2.0-2.3 × faster on a single 24 GB GPU. Use that repo for interactive / single-request workloads where latency matters.

All quants made using imatrix with calibration data v5, the same calibration set bartowski uses for the Qwen3.6 base release — so quality fingerprints are directly comparable to bartowski's Qwen_Qwen3.6-27B-GGUF repo.

Why this merge exists

Same-base DARE-TIES (Omnimerge_v2 method) merge of Qwen/Qwen3.6-27B + 3 Qwen3.6 fine-tunes. Direct successor to ManniX-ITA/Qwen3.5-27B-Omnimerge-v2 on the newer Qwen3.6 base, with mlp.{gate,up,down}_proj copied verbatim from clean Qwen3.6 (the "MLP-passthrough" surgery) to defend against a Qwen3.6-specific reasoning-tag fragility we found during forensic delta inspection. See the v4 model card for the full story, scripts, and benchmark methodology.

Benchmark headline (Q6_K, head-to-head vs Qwen3.6 base + Omnimerge-v2)

All scored under identical llama.cpp + lm_eval conditions (--reasoning-format deepseek --reasoning-budget 8192 --parallel 2, raw /v1/completions, no chat template).

Benchmark	Qwen3.6 base Q6_K (bartowski)	Omnimerge-v2 (Qwen3.5 base)	Omnimerge-v4-MLP (this)	Δ vs base	Δ vs v2
HumanEval pass@1 (164q)	84.76%	79.27%	83.54% (137/164)	−1.22 pp	+4.27 pp
MBPP pass@1 (500q) — corrected*	57.60%	74.60%	73.00% (365/500)	+15.40 pp	−1.60 pp
GPQA Diamond pass@1 (flex) — full greedy§	not measured	69.19% (full 198q)	78.28% (155/198)	—	+9.09 pp

* MBPP scores are post-<think>-stripping (lm_eval's raw scorer SyntaxErrors on literal < in exec(prompt+completion+tests)). See the v4 model card for the per-model recovery breakdown.

§ Canonical full-198q greedy GPQA result measured 2026-05-22 on pod 37268930 (Vast.ai 3090) with the patched eval chain (lm-eval 0.4.11 + max_length=32768 override + the api_models.py:545 UnboundLocalError patch + aiohttp lifecycle workaround). Sampler: do_sample=False, temperature=0.0, max_gen_toks=8192. Wall time 4 h 55 min. Companion strict-match (rigid Answer: X template) is 7.58 % — the model emits CoT verbosely rather than the strict template, so flex is the real quality signal. Earlier card revisions reported an ≈ 84.75 % partial result (177/198 sampled at T=0.6, budget=16384); that number is superseded by this canonical greedy measurement on the full bench — the 6.5 pp difference is driven by the methodology change (sampler / budget / completeness), not by a model change.

Available Quantizations

All 27 files (F16 + 26 imatrix-quantized tiers, ~417 GB total) are uploaded and ready. imatrix.dat (used for every quant) is in the repo root for audit and reproduction.

Quantization	File size	Use case
F16 (full precision)	50.11 GB	Conversion source / lossless reference
Q8_0	26.63 GB	Highest fidelity, large
Q6_K_L	21.14 GB	Q6_K with embed/output at Q8_0
Q6_K	20.57 GB	Recommended high tier — eval methodology used this
Q5_K_L	18.64 GB	Q5_K_M with embed/output at Q8_0
Q5_K_M	17.91 GB	Strong fidelity, balanced
Q5_K_S	17.40 GB	Slightly smaller K-mix
Q4_K_L	16.29 GB	Q4_K_M with embed/output at Q8_0
Q4_1	15.91 GB	Legacy 4-bit, dense
Q4_K_M	15.41 GB	Recommended balanced tier for most users
IQ4_NL	14.72 GB	Importance-aware 4-bit non-linear
Q4_K_S	14.52 GB	K-mix small variant
Q4_0	14.41 GB	Legacy 4-bit
IQ4_XS	14.05 GB	IQ4 extra-small
Q3_K_XL	13.42 GB	Q3_K_L with embed/output at Q8_0
Q3_K_L	13.36 GB	3-bit K-mix large
Q3_K_M	12.39 GB	3-bit K-mix medium
IQ3_M	11.72 GB	Importance-aware 3-bit medium
Q3_K_S	11.24 GB	3-bit K-mix small
IQ3_XS	11.15 GB	IQ3 extra-small
Q2_K_L	11.13 GB	Q2_K with embed/output at Q8_0
IQ3_XXS	10.42 GB	IQ3 extra-extra-small
Q2_K	9.98 GB	2-bit K-mix
IQ2_M	9.32 GB	Importance-aware 2-bit medium
IQ2_S	8.72 GB	IQ2 small
IQ2_XS	8.47 GB	IQ2 extra-small
IQ2_XXS	7.85 GB	IQ2 extra-extra-small (smallest)

How to Use

With llama.cpp:

bash

# Recommended args for reasoning-tag-emitting models:
llama-server \
    -m Qwen3.6-27B-Omnimerge-v4-Q4_K_M.gguf \
    -c 32768 -ngl 99 -t 12 --no-warmup \
    --reasoning-budget 8192

See Reasoning budget and thinking stop phrase below for the budget, the wrap-up phrase that stops reasoning leaking into the answer, and why --reasoning-format plays no part in it. The published evals additionally pin --reasoning-format deepseek so that lm_eval sees only the answer; it is not needed for ordinary serving.

Swap Q4_K_M for any tier from the table above. Q6_K matches the methodology used in our published evals; Q4_K_M is the typical "balanced" choice for most users.

For multimodal (vision) inference: the mmproj projector is in bartowski/Qwen_Qwen3.6-27B-GGUF and works with this model unchanged (vision tower is preserved verbatim from the base).

With ollama: use a Modelfile pointing to one of the GGUFs above, or HF direct load.

Reasoning budget and thinking stop phrase (llama.cpp)

Qwen 3.6 reasons at length by design, and on a hard prompt it can consume the whole context window before it answers. llama.cpp can bound the thinking block with a sampler, and — the part that actually matters — tell the model why the block is being closed.

Needs llama.cpp b8508 or newer for the flags, b10091 or newer for the per-request overrides.

Serve with a bounded thinking block

bash

llama-server -m Qwen3.6-27B-Omnimerge-v4-Q4_K_M.gguf -c 32768 -ngl 99 \
    --jinja \
    --reasoning-budget 8192 \
    --reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n' \
    --temp 0.6 --top-k 20 --top-p 0.95

flag	meaning
`--reasoning-budget N`	`-1` unrestricted (default), `0` close the block immediately, `N > 0` cap it at N tokens
`--reasoning-budget-message`	text written into the block just before the closing tag is forced
`--jinja`	required — the delimiters come from the chat template (`<think>` … `</think>`). Without it llama.cpp has no tags to count and the budget silently does nothing

Both flags also read from the environment: LLAMA_ARG_THINK_BUDGET and LLAMA_ARG_THINK_BUDGET_MESSAGE.

--reasoning-format is not part of this. It only decides how the thinking is handed back — message.reasoning_content versus left inline in message.content — and never whether the budget is enforced: the delimiters the sampler counts are set by the chat template regardless, so the cap binds under auto, deepseek and none alike. The default auto already extracts reasoning and is behaviourally identical to deepseek (they differ only in name; the sole branch in the parser is != none). Leave it at the default so the model's own tool-call and channel handling stays in play, and pin deepseek only when a harness needs the thinking kept out of content.

--reasoning-budget on its own forces the closing tag the moment the budget runs out, wherever the model happens to be. When that lands mid-thought the model frequently does not register that it was interrupted: it carries on reasoning, now inside the visible answer. The stop phrase is what prevents that — it gives the model a reason to be finishing.

Two wordings that work

bash

# "qwen" — the string Qwen's own service uses, from their docs
--reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n'

# "voice" — shorter, in the model's own reasoning voice
--reasoning-budget-message $'\n\nOK, I have enough to answer now.\n'

Wording is model-specific: Qwen note that the ability to act on such a message "is not explicitly trained but emerges naturally", so it is worth trying both on your own workload. Leading and trailing newlines matter — they keep the phrase off whatever half-finished line the cut landed on.

What it measures out to

Measured on the Qwen3.6-35B-A3B base this model is pruned from. Three hard questions, temperature 0.6, fixed seed, answer characters with wall time in brackets. Every run answered all three correctly, and thinking length is unchanged by the message in every row:

budget	no message	`qwen`	`voice`
2048	1907 (69 s)	1838 (42 s)	1615 (41 s)
4096	18015 (170 s)	2642 (78 s)	1441 (104 s)
8192	3642 (158 s)	1848 (129 s)	2023 (175 s)

The 4096 row is the failure this exists for: the cap lands mid-thought and the reasoning simply continues in the answer, ten times longer and 2.2x the wall time, for the same three correct answers. Both phrases remove it.

Per request, instead of per server

The server accepts both as request fields, overriding the command line:

json

{
  "messages": [ ... ],
  "thinking_budget_tokens": 8192,
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On the raw /completion endpoint the delimiters are not inferred, so they have to be supplied with the budget:

json

{
  "prompt": "...",
  "reasoning_budget_tokens": 8192,
  "reasoning_budget_start_tag": "<think>",
  "reasoning_budget_end_tag": "</think>",
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On b10091 the message field must be present on /completion requests even when empty: llama.cpp builds the sequence it forces from message + end_tag inside that field's handler, so omitting it leaves the budget with nothing to force — the sampler logs as though the cap fired while the thinking block stays open.

Rules of thumb

Keep -c several times larger than the budget. A budget equal to the context lets the thinking phase fill the window on its own.
A quarter of the context is a sensible starting point: 8192 at -c 32768.
Qwen recommend keeping a thinking budget above 1024 tokens; below that the cap tends to land before the model has committed to an approach.
The budget is per thinking block, not per response — the sampler re-arms when it sees a new opening tag, so a multi-turn agent gets a fresh window each time.

imatrix.dat

The imatrix.dat (~14 MB) used to generate every quant in this repo is uploaded alongside the GGUFs at the repo root. Reproducible, auditable.

Reproducing

See scripts/ on the source v4 model repo:

dare_ties_merge.py — main merger (auto-detects Qwen3.6 base via output_gate_type and applies MLP-skip)
v4_mlp_passthrough.py — post-process: rebuild merged dir with MLP layers from base
quantize_gguf.py — the script that built this repo

For dense (non-Gemma-4-MoE) models, pass --exclude CD-Q6_K,CD-Q5_K_M,CD-Q4_K_M,CD-Q3_K_M,CD-Q2_K to skip ContribDynamic tiers (those require Gemma 4 expert-contribution maps).

License

Apache-2.0 (inherited from Qwen/Qwen3.6-27B and the fine-tune sources).

Acknowledgements

Qwen team for the Qwen3.6 base
rico03, ValiantLabs, kai-os for the fine-tunes
bartowski for the calibration_datav5.txt set used here
DARE / TIES / DARE-TIES authors and the arcee-ai/mergekit community

shaffhausen/Qwen3.6-27B-Omnimerge-v4-GGUF

Qwen3.6-27B-Omnimerge-v4-GGUF

Why this merge exists

Benchmark headline (Q6_K, head-to-head vs Qwen3.6 base + Omnimerge-v2)

Available Quantizations

How to Use

Reasoning budget and thinking stop phrase (llama.cpp)

Serve with a bounded thinking block

Two wordings that work

What it measures out to

Per request, instead of per server

Rules of thumb

imatrix.dat

Reproducing

License

Acknowledgements

shaffhausen/
Qwen3.6-27B-Omnimerge-v4-GGUF