GGUF quantizations of CMKL/MANGO1.5-Qwen3.5-9B, CMKL University's dense 9B bilingual (Thai/English) fine-tune of Qwen/Qwen3.5-9B, for use with llama.cpp and any runtime built on it (Ollama, LM Studio, koboldcpp, text-generation-webui, etc.).

See the base model card for full details on training data, evaluation results, and intended use. This repo only covers the GGUF conversion/quantization itself.

About this conversion

Source: merged full-precision (bf16) checkpoint, converted with a llama.cpp build carrying native support for the Qwen3.5 hybrid linear-attention (Gated DeltaNet) architecture (general.architecture = qwen35).
Scope: text-only. The base Qwen3.5 architecture is natively multimodal (it ships a vision tower), but this fine-tune was trained text-only and the vision tower is not included in these GGUFs — there is no accompanying mmproj file, and these files will not do image/video input.
Speculative decoding (MTP): the source checkpoint includes Qwen3.5's Multi-Token-Prediction (NextN) draft layer, and it is included here (qwen35.nextn_predict_layers = 1, 33 blocks total: 32 main + 1 MTP). Whether your runtime actually uses it for self-speculative decoding depends on that runtime's support for qwen35's NextN layer — check your llama.cpp build's release notes if in doubt.
Vocab / chat template: the Qwen3.5 BPE tokenizer and Jinja chat template are embedded in the GGUF metadata, including the bilingual system-prompt default and <think>/<tool_call> hybrid-reasoning template from the base model.

Files

File	Quant	Size	Notes
`MANGO1.5-Qwen3.5-9B-BF16.gguf`	BF16	18 GB	Full precision, lossless vs. source. Use as a base for your own requantization, or if you have the VRAM/RAM and want the highest fidelity.
`MANGO1.5-Qwen3.5-9B-Q8_0.gguf`	Q8_0	9.2 GB	Near-lossless. Best choice if quality matters more than size and you have the room.
`MANGO1.5-Qwen3.5-9B-Q6_K.gguf`	Q6_K	7.1 GB	Very close to Q8_0 quality, meaningfully smaller.
`MANGO1.5-Qwen3.5-9B-Q5_K_M.gguf`	Q5_K_M	6.2 GB	Good middle ground; low measurable quality loss.
`MANGO1.5-Qwen3.5-9B-Q4_K_M.gguf`	Q4_K_M	5.4 GB	Recommended default. The community-standard balance of quality vs. size/speed; fits comfortably on 8GB-class GPUs or in ~6-8GB RAM for CPU inference.
`MANGO1.5-Qwen3.5-9B-Q4_K_S.gguf`	Q4_K_S	5.2 GB	Slightly smaller/faster than Q4_K_M at a small further quality cost. Use if you're VRAM-constrained.
`MANGO1.5-Qwen3.5-9B-IQ4_XS.gguf`	IQ4_XS	5.0 GB	Smallest 4-bit-class option here. Built without an importance matrix, so treat it as a size-first pick rather than the best-quality 4-bit option — Q4_K_S/Q4_K_M are the safer default at a similar size.

All files were produced from the same BF16 base with llama-quantize; no calibration/importance-matrix data was used for any of them (including IQ4_XS), so IQ4_XS in particular may underperform its usual reputation relative to the K-quants.

Which one should I use?

Not sure / general use: Q4_K_M.
Have 12GB+ VRAM and want better quality: Q5_K_M or Q6_K.
Want maximum fidelity / plan to requantize yourself: BF16.
Squeezed for space (e.g. 6-8GB VRAM): Q4_K_S, then IQ4_XS if you still need to shave more off.

How to use

llama.cpp

bash

# CLI, one-shot
llama-cli -m MANGO1.5-Qwen3.5-9B-Q4_K_M.gguf -ngl 99 -p "Hello, who are you?"

# OpenAI-compatible server
llama-server -m MANGO1.5-Qwen3.5-9B-Q4_K_M.gguf -ngl 99 --port 8080

-ngl 99 offloads all layers to GPU; drop it (or lower the number) for CPU-only or partial offload. The chat template is embedded, so llama-cli's interactive/conversation mode and llama-server's /v1/chat/completions endpoint will apply it automatically — no need to hand-format the bilingual system prompt from the base model card, though you're free to override it per-request.

Ollama

Ollama is llama.cpp-based, so it's subject to the same architecture-support requirement below (recent Ollama build needed for qwen35).

Directly from this Hub repo (once uploaded — see note below), no local download step needed:

ollama run hf.co/CMKL/MANGO1.5-Qwen3.5-9B-GGUF:Q4_K_M

Swap Q4_K_M for any other quant tag from the Files table above.

From a GGUF file you already have locally, via a Modelfile:

bash

cat <<'EOF' > Modelfile
FROM ./MANGO1.5-Qwen3.5-9B-Q4_K_M.gguf
EOF

ollama create mango1.5 -f Modelfile
ollama run mango1.5

No TEMPLATE directive is needed in the Modelfile — Ollama reads the chat template embedded in the GGUF metadata (same one llama-cli/llama-server use) automatically.

Requires a recent llama.cpp / Ollama build

Qwen3.5's hybrid linear-attention (qwen35 architecture) support is recent. If your llama.cpp build (or the llama.cpp version bundled inside your Ollama install) predates it, these files will fail to load with an unknown-architecture error — update to a current build (e.g. ghcr.io/ggml-org/llama.cpp:full-cuda / server-cuda on Docker Hub / GHCR, or build from source; ollama --version / ollama pull the latest release for Ollama) before reporting a loading issue.

License

Inherits the Apache 2.0 license of the base model. See Qwen/Qwen3.5-9B for exact terms.

Credits

Fine-tune: CMKL/MANGO1.5-Qwen3.5-9B — Atikan Payungthong, Dr. Akkarit Sangpetch (CMKL University)
Base model: Qwen/Qwen3.5-9B
Quantization tooling: llama.cpp