The hybrid quant employs different quantization levels on a per layer basis to enable
both high performance and small file size at the same time. The quants employed are all K to avoid
slow CPU or older GPU processing of IQ quants. An extended layer definition E quant Q4_E_H for the model is defined as follows (updated 8/24/2026):
The quant was optimized for approximately Q4_K_M bpw with strong performance across a curated set of reasoning test prompts,
scoring solid 100% across the test set with greedy deterministic sampling, exhibiting good common sense. The model exhibits
severe overthinking on some prompts but no infinite rep loops were found on any of the prompts. The quant includes layer 64 nextn MTP layer.
If not using MTP the loader will give warning messages about unused tensors on layer 64 but the model will run correctly.
The quant was updated 8/24/2026 for minium quant across layers at Q4_K to reduce entropy loss and provide noticeable performance
improvement compared to the first Q4_E_H quant.
Comparison:
Quant
size
PPL
Comment
Q4_K_M
16.5e9
10.6
Q4_K_M with default embedding and output (no MTP layer)
Qwen3.6-27B is a vision capable dense RL model. It can be used together with its multimedia projector layers to process images and text inputs
and generate text outputs. The mmproj file is made available in this repository.
Due to the attention scheme used in the model, straightforward speculation approaches cannot be used. Later versions of llama.cpp as of 4/23/2026 attempt to
circumvent this problem however efficiency will not be good so use of speculation prior to Qwen3.5 MTP patches is not recommended.
As of llama.cpp b9180 MTP support for the model was added to upstream and may be experimented with by using the Q4_E_H_MTP quant.
On a 2x 4070 setup (1 RPC) approx performance is:
Q
QKV
NKV
gen tps
Q4_E_H
F16
100k +
25
Q4_E_H
Q8_0
160k +
23
The model went into an infinite rep loop with with Q8_0 QKV on one test prompt. No instability was found for F16 QKV.
The model appears to be trained to decide itself whether to do a think block or not. When it does a think block it can fall into very
heavy overthinking but does come up with accurate answers. Over a small set of eval prompts the model did extremely well. To avoid
the overthinking inject think start and think stop tokens first thing after assistant prompt:
If the model doesnt feel like doing thinking on a given prompt it will automatically do this. To force the model into
a think block inject a bootstrap think start after the assistant prompt:
"<think>\n"
The model was found to be highly capable on reasoning tasks when skipping think block, with zero overthinking, just accurate
direct deductions to final solutions. On two test prompts it was necessary to force the model into think mode to get the
right answer.
Note: testing shows this model is prone to severe overthinking on some prompts, significantly higher than Qwen 3.5-27B.
This version of the model appears to have been fine tuned with stronger forced introspections which make it harder for it
to escape the think block if there is even the slightest ambiguity in its reasoning.
VISION:
The quant went 2 for 2 on a couple tough bird ID images prompts with severe overthinking on one
of the images where it toggled back and forth between two candidates for a long time.
CODE:
The quant was tested across a small set of code gen prompts and found to be very solid in its ability to generate
working programs.
