llama-quantize \
  --imatrix imatrix.dat \
  --tensor-type blk.64.attn_k.weight=q4_K \
  --tensor-type blk.64.attn_q.weight=q4_K \
  --tensor-type blk.64.attn_output.weight=q4_K \
  --tensor-type blk.64.ffn_down.weight=q4_K \
  --tensor-type blk.64.ffn_gate.weight=q4_K \
  --tensor-type blk.64.ffn_up.weight=q4_K \
  --tensor-type blk.64.nextn.eh_proj.weight=q4_K \
  Qwen3.6-27B-Omnimerge-v4-MTP-F16.gguf \
  Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS.gguf \
  IQ3_XXS

Running with MTP

bash

llama-server \
  -m Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2

HumanEval

Evaluated using:

lm-evaluation-harness
Raw OpenAI-compatible /v1/completions
Temperature = 0
Model-specific tokenizer (ManniX-ITA/Qwen3.6-27B-Omnimerge-v4)
llama.cpp CUDA build b9632

Model	Tokenizer	HumanEval pass@1	Size
Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS	ManniX-ITA/Qwen3.6-27B-Omnimerge-v4	80.49% ± 3.10%	10.6 GB
ggufbench/Qwen3.6-27B-4bpw-16GB-VRAM	unsloth/Qwen3.6-27B	81.10% ± 3.07%	12.58 GB
unsloth/Qwen3.6-27B-MTP-GGUF:IQ2_M	unsloth/Qwen3.6-27B	71.95% ± 3.52%	10.2 GB

The IQ3_XXS Omnimerge quant is within approximately one HumanEval problem of the larger 4bpw/16GB-VRAM quant in this setup, while remaining much closer in size to the 10GB-class IQ2_M quant.

Evaluation Command

bash

python -m lm_eval \
  --model local-completions \
  --model_args "model=Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS,base_url=http://127.0.0.1:1234/v1/completions,tokenizer=ManniX-ITA/Qwen3.6-27B-Omnimerge-v4,tokenizer_backend=huggingface" \
  --tasks humaneval \
  --batch_size 1 \
  --confirm_run_unsafe_code

Test Hardware

Google Colab
NVIDIA L4 (24 GB VRAM)
Context length: 65,536
MTP enabled

Notes

MTP functionality is preserved.
Code-completion benchmarks such as HumanEval should use raw /v1/completions rather than chat-completions.
Benchmark results are sensitive to tokenizer selection; use the Omnimerge tokenizer for reproducible results.

Credits

Base model: ManniX-ITA
Quantization: kairusama
Runtime: llama.cpp

kairusama/Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS-GGUF

Qwen3.6-27B-Omnimerge-v4-MTP IQ3_XXS GGUF

Quantization