Original: 70GB BF16 → Quantized: 20GB Q4_K_M
HumanEval: 95.1% (164 problems) — beats Claude Opus 4.6 (92%) and GPT-4.1 (90.2%)
Active Parameters: 3B per token (MoE architecture)
Min GPU: 24GB VRAM (RTX 4090, L4, A6000)
Speed: ~109 tok/s on H100, ~30-50 tok/s on consumer GPUs

Usage with Ollama

Download the GGUF file, then create a Modelfile:

FROM ./qwen3.6-35b-a3b-q4_k_m.gguf
PARAMETER temperature 0.7
PARAMETER stop <|im_end|>
TEMPLATE """{{- range .Messages }}
{{- if eq .Role "system" }}<|im_start|>system
{{ .Content }}<|im_end|>
{{ else if eq .Role "user" }}<|im_start|>user
{{ .Content }}<|im_end|>
{{ else if eq .Role "assistant" }}<|im_start|>assistant
{{ .Content }}<|im_end|>
{{ end }}
{{- end }}<|im_start|>assistant
<think>

</think>

"""

Then run:

bash

ollama create qwen36 -f Modelfile
ollama run qwen36

The template includes an empty think block to disable thinking mode (direct answers). Remove the think tags to enable chain-of-thought reasoning.

Benchmark Results

Model	HumanEval (pass@1)
This model (Q4_K_M, 20GB)	95.1%
claw-lite Enhanced (BF16 + self-refinement)	94.5%
Claude Opus 4.6	92.0%
Qwen 3.6 35B-A3B BF16 (stock, 70GB)	91.5%
GPT-4.1	90.2%
Claude Sonnet 4.6	88.0%
DeepSeek-V3	82.6%
Qwen3 235B-A22B (7x larger)	80.4%
Llama 3.3 70B	75.0%

Quantization Details

Method: llama.cpp Q4_K_M (4.88 bits per weight)
Original size: 66,152 MiB (16.01 BPW)
Quantized size: 20,176 MiB (4.88 BPW)
Compression: 3.3x
Source: Qwen/Qwen3.6-35B-A3B (BF16)

notanoption1/Qwen3.6-35B-A3B-Q4_K_M-GGUF

Qwen 3.6 35B-A3B Q4_K_M GGUF

Key Stats

Usage with Ollama

Benchmark Results

Quantization Details

notanoption1/
Qwen3.6-35B-A3B-Q4_K_M-GGUF