Qwen3.8-27B GGUF — Quantized by BatiAI

Qwen's newest 27B — on your Mac, in Korean, in one line.

Weights dropped 2026‑08‑13. Quantized the same week, straight from the official release. Six sizes from 10 GB, vision projector included, Korean output actually tested — and Ollama tags published alongside, which most GGUF drops skip.

🧠 What Qwen3.8-27B actually is

📦 Pick your size

⚠️ Q3_K_M is smaller and slower on Apple Silicon. Skip it if you can.

Size ordering does not predict speed here. Measured on M4 Max (Metal, -ngl 99):

Quant	Size	Generation
Q3_K_M	12.38 GiB	19.2 t/s ← smaller
IQ4_XS	14.04 GiB	22.4 t/s (+17%)
Q4_K_M	15.40 GiB	22.7 t/s (+18%)

Why: generation speed is capped by memory bandwidth ÷ model size — every token reads the whole model. Against that ceiling (M4 Max, 546 GB/s), the efficiency numbers show what's really going on:

Quant	Ceiling	Measured	Efficiency
Q2_K_S	53.4 t/s	28.2	53%
IQ3_XXS	48.8	23.7	48%
Q3_K_M	41.1	19.2	47%
IQ4_XS	36.2	22.4	62%
Q4_K_M	33.0	22.7	69%

The low-bit tiers sit at ~50% of their ceiling while Q4_K_M reaches 69%. They aren't bandwidth-limited — they're compute-limited on dequantization. Q2_K/Q3_K/IQ3 unpack in more steps and that costs Metal ALU time. So shrinking the file past Q4 buys you memory, not speed.

On CUDA the same quants land at 70–80% efficiency across the board (50.8 / 50.3 / 46.2 t/s — effectively tied), so this is an Apple Silicon effect. Linux/CUDA users can pick on size alone.

And it isn't specific to this model. Going back through our own benchmark archive, the same Q3_K_M → Q4_K_M ordering shows up across four unrelated model families and two different chips:

Model	Chip	Q3_K_M	Q4_K_M	Gap
Granite 4.1 (8B dense)	M4 Max	60.8	77.4	+27%
Qwen3.8‑27B (this model)	M4 Max	19.2	22.7	+18%
Gemma 4 26B (MoE)	M4 Max	78.0	87.6	+12%
Granite 4.1 (8B dense)	M4 (mini)	12.8	14.3	+11%
Qwen3.6‑27B	M4 Max	15.3	16.6	+8%

Dense and MoE, 8B and 27B, Max and non‑Max — same direction every time. Treat it as a property of Metal's Q3_K kernel, not of any one checkpoint.

_{Rows other than Qwen3.8‑27B were measured through Ollama (bench.sh); the Qwen3.8‑27B row is llama-bench. Absolute values aren't comparable across harnesses — the ordering within each row is the finding. Qwen3.8‑27B numbers: llama.cpp b9430, M4 Max 128 GB, idle machine. The reversal was re‑confirmed on b10450.}

Practical reading: on a Mac, Q4_K_M is the most efficient kernel and IQ4_XS is within noise of it — pick IQ4_XS to save 1.4 GB, Q4_K_M if you have the room.

🚨 16 GB Macs: this model does not fit. Please don't waste the download.

We originally listed 16 GB targets here. That was wrong and we're correcting it. A 16 GB Mac has roughly 12 GB usable after macOS, and measurement on an M4 Max shows q2 — the smallest quant — sitting at a 14 GB floor even with context squeezed to 4096:

num_ctx	Memory (q2, measured)
262144 (model default)	32 GB
32768	16 GB
8192 / 4096	14 GB (floor)

The weights alone are 10 GB; KV cache and runtime overhead do the rest. On 16 GB you get swapping, partial CPU fallback, and a model that feels broken. For 16 GB machines we recommend Qwen3.6‑35B‑A3B (MoE, 3B active) or a smaller dense model instead.

Context length is the biggest memory lever here — this checkpoint's native context is 262,144, so the default allocation is enormous. Our Ollama tags ship a modest num_ctx default; with llama.cpp set -c explicitly rather than inheriting the model default.

_{Measured on M4 Max 128 GB by the BatiFlow Mac team, Ollama 0.20.0. Numbers for 24/32/36/48 GB rows are extrapolated from that floor plus quant size — we'll replace them with direct measurements as they come in.}

All six were rebuilt on 2026‑08‑15. Our first build mishandled this checkpoint's MTP layer — two files wouldn't load and every Ollama tag failed to start. We found it while benchmarking, pulled the broken files, and republished all six with the fix. Current files are verified on both engines. What went wrong and the correct flags are in "Two things that will bite you" below.

✅ Verified — captured from this build

[산수]
185

[한국어]
한국의 수도는 서울입니다.
비빔밥은 밥에 각종 채소와 고추장을 넣고 비벼 먹는 한국 대표 음식입니다.
불고기는 달콤한 양념에 재운 고기를 구워 먹는 인기 있는 한국 요리입니다.
떡볶이는 떡에 매콤달콤한 양념을 버무려 먹는 대표적인 한국 길거리 음식입니다.

[툴콜]
{"tool":"get_weather","args":{"city":"부산"}}

[코딩]
```python
def binary_search(arr, target):
    left, right = 0, len(arr) - 1
    while left <= right:
        mid = (left + right) // 2
        if arr[mid] == target:
            return mid
        elif arr[mid] < target:
            left = mid + 1
        else:
            right = mid - 1
    return -1
```

Why we print this. A quantized model can look fine on English benchmarks and fall apart in Korean — we've measured models that repeat a single word 60 times. So we run Korean, tool‑calling and code through every build we publish, and paste what came out. Good or bad.

🚀 Usage

Ollama (simplest)

llama.cpp

Vision (image input)

⚠️ Two things that will bite you (we hit both)

1. Thinking mode eats your token budget

2. blk.64 is an MTP layer — and pruning it is a trap

⚡ Speed (measured, not estimated)

Want it meaningfully faster? Change the model, not the quant.

✨ What BatiAI did differently

📜 License — Apache 2.0

🔗 Related

• Previous generation: batiai/Qwen3.6-27B-GGUF · batiai/Qwen3.5-27B-GGUF
• MoE sibling: batiai/Qwen3.6-35B-A3B-GGUF
• Base model: Qwen/Qwen3.8-27B