This repository contains custom-quantized, text-only configurations of the Qwen3.6 model family, optimized explicitly for local repository-level agentic coding on Apple Silicon using the oMLX inference engine.

🎯 Why This Was Created

Standard open-weight releases and community pre-converted MLX files (like standard Unsloth) routinely strip out Multi-Token Prediction (mtp.*) tensors during conversion. While this saves minimal disk space, it completely disables the hardware-level Native MTP decoding pipeline on Apple Silicon.

This project was built to solve specific long-context constraints encountered on a MacBook Pro M2 Max (96GB Unified Memory, 400 GB/s memory bandwidth):

The Dense Speed Bottleneck: Dense architectures like the Qwen3.6-27B provide significantly higher coding accuracy, sharper reasoning, and far fewer repetitive loops than sparse MoE setups. However, they naturally run much slower. Preserving the original MTP heads unlocks a ~1.5x generation speedup to make dense coding completely viable locally.
Overcoming the 128k Context Wall: Traditional backends like LM Studio frequently experience severe latency degradation or choke entirely when context windows approach 128k tokens due to unoptimized KV cache regeneration. Moving to oMLX's specialized two-tier caching eliminates this overhead.
Maximized Precision: While 4-bit and 5-bit quants are popular for saving memory, a 96GB unified memory architecture provides plenty of overhead. These builds use oQ8 (8-bit) precision to guarantee lossless adherence to complex syntax, indentation, and nested tool calls.

🚀 Key Differences

Feature / Attribute	Standard Qwen3.6 / Unsloth	This Custom Build (`oQ8-fp16-mtp`)
Native MTP Heads	Stripped out during conversion	Fully Preserved (Enables Native MTP toggle)
Vision Model (VLM)	Included by default (wastes memory space)	Stripped (Text/Coding only; ~2-3% smaller)
Quantization Method	Standard Uniform / GGUF Dynamic	oQ8 (Dynamic mixed-precision calibration)
Non-Quant Weight DType	`bfloat16`	`float16` (~20% faster prefill on M1/M2 Silicon)

💻 Hardware & RAM Recommendations

Mac Hardware Configuration	RAM Recommendation	Status / Performance Expectation
M1 Max / M2 Max / M3 Max (Base 32GB/36GB/48GB)	48GB Unified Memory	Supported (Minimal) — Good for standard queries, but context window will be limited.
M1 / M2 / M3 / M4 Max / Ultra	64GB Unified Memory	Recommended — Great performance, handles deep context windows (~128k) comfortably.
M2 Max / M3 Max (96GB / 128GB)	96GB / 128GB Unified Memory	Optimal / Best Experience — The user's M2 Max 96GB is ideal. Allows running maximum context (262k) with zero slowdowns and plenty of system overhead.

🛠️ Quantization Settings

This model was quantized using oMLX v0.4.3 with the following settings:

Source Model: Qwen/Qwen3.6-27B
Sensitivity Model: None (use source model)
oQ Level: oQ8
Text Only: Enabled (Excludes vision encoder weights, saving ~2-3% memory footprint)
Preserve MTP weights: Enabled (Preserves mtp.* tensors for Native MTP support)
Non-quant weight dtype: float16 (Provides ~20% faster prefill on M1/M2 Apple Silicon)

⚙️ Optimized oMLX Settings (v0.4.3)

To replicate the ultra-long context development workspace used with OpenCode 1.16.2, apply the following server specifications in your oMLX dashboard:

Model Basic Settings

Reasoning Parser: qwen_3_coder (Isolates CoT thoughts to protect IDE syntax)
CTX Window: 262,144
Max Tokens: 32,768
Temperature: 0.6
Top P / Top K: 0.95 / 20
Min P: 0
Repetition / Presence Penalty: 1 / 0

Model Advanced Settings

Enabled Thinking: Checked (True)
Chat Template Kwargs: enable_thinking: true, preserve_thinking: true
Native MTP: Checked (True)

Resource Management & Cache

Memory Guard: Aggressive (Enforces strict macOS memory/swap cleanup)
Hot Cache Limit (RAM): 40GB (Allocated for high-speed Unified Memory history)
Cold Cache Limit (SSD): 371GB (Serialized safetensors storage for context overflow)
Max Concurrent Requests: 2 (Prevents splitting the 400 GB/s bandwidth bus)
Embedding Batch Size: 32
Chunked Prefill: Enabled (Prevents instantaneous out-of-memory crashes on large files)
Burst Decode: Aggressive (Coalesces tokens for maximum raw typing speed)
Initial Cache Blocks: 256
SSE Keepalive Mode: Chunk

🌡️ Thermal Optimization Notice

Sustained local execution across deep context windows will heavily tax the Apple Silicon GPU/CPU, leading to internal heat buildup. Because Apple's default fan profiles prioritize silence over longevity, they delay high-RPM cooling until thermal throttling is already occurring.

To manage this proactively during extended coding agent sessions, use smc-command—an adapted CLI fan utility tailored to trigger more aggressive fan curves based on exact thermal sensors.

Source Repository: smc-command on Codeberg

oMLX Benchmark - Performance

oMLX - LLM inference, optimized for your Mac
https://github.com/jundot/omlx
Benchmark Model: Qwen3.6-27B-oQ8-fp16-mtp
================================================================================

Single Request Results
--------------------------------------------------------------------------------
Test                TTFT(ms)    TPOT(ms)        pp TPS        tg TPS      E2E(s)    Throughput    Peak Mem
pp1024/tg128          5468.7       70.23   187.2 tok/s    14.4 tok/s      14.388    80.1 tok/s    28.81 GB
pp4096/tg128         20370.5       61.99   201.1 tok/s    16.3 tok/s      28.243   149.6 tok/s    30.26 GB
pp8192/tg128         41398.6       54.51   197.9 tok/s    18.5 tok/s      48.321   172.2 tok/s    30.89 GB
pp16384/tg128        85002.6       56.29   192.7 tok/s    17.9 tok/s      92.152   179.2 tok/s    32.14 GB
pp32768/tg128       180387.9       57.55   181.7 tok/s    17.5 tok/s     187.697   175.3 tok/s    34.64 GB
pp65536/tg128       409535.7       62.85   160.0 tok/s    16.0 tok/s     417.518   157.3 tok/s    39.66 GB
pp200000/tg128     1900213.2       89.74   105.3 tok/s    11.2 tok/s    1911.611   104.7 tok/s    56.53 GB

Continuous Batching
pp1024 / tg128
--------------------------------------------------------------------------------
Batch           tg TPS   Speedup        pp TPS    pp TPS/req    TTFT(ms)      E2E(s)
1x          14.4 tok/s     1.00x   187.2 tok/s   187.2 tok/s      5468.7      14.388
2x          23.2 tok/s     1.61x    50.4 tok/s    25.2 tok/s     34893.7      51.639
4x          46.3 tok/s     3.22x   127.0 tok/s    31.8 tok/s     21303.2      43.311
8x          92.5 tok/s     6.42x   108.4 tok/s    13.6 tok/s     42968.2      86.633

Noctalin/Qwen3.6-27B-oQ8-fp16-mtp

Qwen3.6 oQ8 Text-Only with Native MTP (Optimized for Apple Silicon)