Tensor type	Quantization	Count
`attn_out.weight`, `attn_qkv.weight`, `ffn_up.weight`, `mm.0.weight`, `mm.2.weight`	Q8_0	83
`ffn_down.weight`	F16	27
Biases, norms, embeddings, positional	F32	224
Total		334

File size: ~601 MB (vs ~889 MB BF16, ~400 MB full Q8_0)
Aligned: yes

Cross-compatibility

Clip metadata and tensor shapes are identical between Qwen3.5-27B and Qwen3.6-27B mmproj files. This file works with both architectures in llama.cpp.

clip.vision.image_size       = 768
clip.vision.patch_size       = 16
clip.vision.embedding_length = 1152
clip.vision.feed_forward_length = 4304
clip.vision.block_count      = 27
clip.vision.head_count       = 16
clip.vision.projection_dim   = 5120
clip.projector_type          = qwen3vl_merger

Usage

bash

llama-server \
  --model Qwen3.6-27B-UD-Q4_K_XL.gguf \
  --mmproj Qwen3.6-27B-mmproj-hybrid-Q8_0-F16.gguf \
  --ctx-size 131072 \
  --n-gpu-layers 65 \
  --cache-type-k q4_0 \
  --jinja \
  --skip-chat-parsing

Tested on

GPU: RTX 3090 24GB
llama.cpp: build b5200+
Model: DAXZEIT/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled-UD-Q4_K_XL-gguf
Vision: functional (image OCR, multimodal reasoning)

Conversion script

python

"""
Convert mmproj-BF16.gguf to hybrid Q8_0/F16 quantization.
Source: unsloth/Qwen3.6-27B-GGUF
"""
import sys, numpy as np
from pathlib import Path
sys.path.insert(0, '/path/to/llama.cpp/gguf-py')
from gguf import GGUFReader, GGUFWriter, GGMLQuantizationType
from gguf.quants import quantize
from gguf.constants import GGUFValueType

SRC = 'mmproj-BF16.gguf'
DST = 'Qwen3.6-27B-mmproj-hybrid-Q8_0-F16.gguf'
Q8_BLOCK_ELEMS, Q8_BLOCK_BYTES = 32, 34

def target_type(name):
    if name.endswith('ffn_down.weight'): return GGMLQuantizationType.F16
    if any(name.endswith(s) for s in (
        'attn_out.weight','attn_qkv.weight','ffn_up.weight','mm.0.weight','mm.2.weight')):
        return GGMLQuantizationType.Q8_0
    return GGMLQuantizationType.F32

def get_f32(tensor):
    if tensor.tensor_type == GGMLQuantizationType.F32: return tensor.data
    raw = tensor.data.view(np.uint16)
    return (raw.astype(np.uint32) << 16).view(np.float32)

def to_q8_0(data_f32):
    orig_shape = data_f32.shape
    flat = data_f32.flatten()
    quantized = quantize(flat, GGMLQuantizationType.Q8_0)
    if len(orig_shape) == 1: return quantized.reshape(1, -1)
    H = int(np.prod(orig_shape[:-1]))
    W_bytes = orig_shape[-1] // Q8_BLOCK_ELEMS * Q8_BLOCK_BYTES
    return quantized.reshape(H, W_bytes)

reader = GGUFReader(SRC)
writer = GGUFWriter(DST, arch='clip')

# Copy scalar metadata
for key, field in reader.fields.items():
    if key.startswith('GGUF.'): continue
    # ... (see full script in convert_mmproj_hybrid.py)

# Arrays — must be added explicitly
writer.add_array('clip.vision.image_mean', [0.5, 0.5, 0.5])
writer.add_array('clip.vision.image_std',  [0.5, 0.5, 0.5])
writer.add_array('clip.vision.is_deepstack_layers', [False] * 27)
writer.add_string('general.name', 'Qwen3.6 27B mmproj hybrid Q8_0-F16')

for tensor in reader.tensors:
    ttype = target_type(tensor.name)
    data_f32 = get_f32(tensor)
    if ttype == GGMLQuantizationType.F32:
        writer.add_tensor(tensor.name, np.ascontiguousarray(data_f32), raw_dtype=ttype)
    elif ttype == GGMLQuantizationType.F16:
        writer.add_tensor(tensor.name, np.ascontiguousarray(data_f32.astype(np.float16)), raw_dtype=ttype)
    elif ttype == GGMLQuantizationType.Q8_0:
        writer.add_tensor(tensor.name, to_q8_0(data_f32), raw_dtype=ttype)

writer.write_header_to_file()
writer.write_kv_data_to_file()
writer.write_tensors_to_file()
writer.close()

License

Apache 2.0 — same as the source model.

DAXZEIT/Qwen3.6-27B-mmproj-hybrid-Q8_0-F16-gguf

Qwen3.6-27B mmproj — Hybrid Q8_0/F16 GGUF

Source

Quantization

Cross-compatibility

Usage

Tested on

Conversion script

License

DAXZEIT/
Qwen3.6-27B-mmproj-hybrid-Q8_0-F16-gguf