You can deploy this model and start earning money today!

sandeshrajx/
Qwen3.6-27B-MTP-Q3_K_S-GGUF

Name: Qwen3.6-27B-MTP-Q3_K_S-GGUF
Availability: InStock

Views

No views yet

Qwen3.6-27B-MTP Q3_K_S GGUF

This is a plain Q3_K_S GGUF quantization of Qwen3.6-27B that preserves the MTP (Multi-Token Prediction) tensors from Radamanthys11/Qwen3.6-27B-MTP-Q8_0-GGUF. It was requantized from the Q8_0 GGUF source with ik_llama.cpp using --allow-requantize.

Requirements

Use ik_llama.cpp for inference. Pass MTP flags to use speculative decoding:

llama-server -m Qwen3.6-27B-MTP-Q3_K_S.gguf -mtp --draft-max 1 --draft-p-min 0.0

Quantization Recipe

This quant was made from Q8_0 rather than directly from fp16, so a small amount of accuracy may have been lost before this Q3_K_S pass. No imatrix was generated or used for this normal K-quant.

bash

custom="blk\.64\..*\.weight=q8_0,blk\..*\.ssm_alpha\.weight=q6_0,blk\..*\.ssm_beta\.weight=q6_0,blk\..*\.ssm_out\.weight=q6_0,token_embd\.weight=q6_0,output\.weight=q8_0"

./ik_llama.cpp/build/bin/llama-quantize \
  --allow-requantize \
  --custom-q "$custom" \
  ./Qwen3.6-27B-MTP-Q8_0.gguf \
  ./Qwen3.6-27B-MTP-Q3_K_S.gguf \
  Q3_K_S 16

Custom tensor overrides:

blk.64.*.weight MTP/nextn tensors: Q8_0
ssm_alpha.weight, ssm_beta.weight, and ssm_out.weight: Q6_0
token embeddings: Q6_0
output tensor: Q8_0
normal repeating tensors: global Q3_K_S

Source

Source Q8_0 GGUF: Radamanthys11/Qwen3.6-27B-MTP-Q8_0-GGUF

sandeshrajx/Qwen3.6-27B-MTP-Q3_K_S-GGUF

Qwen3.6-27B-MTP Q3_K_S GGUF

Requirements

Quantization Recipe

Source

sandeshrajx/
Qwen3.6-27B-MTP-Q3_K_S-GGUF