Qwen3.6-35B-A3B oQ5 Text-Only with Native MTP (Optimized for Apple Silicon)
This repository contains custom-quantized, text-only configurations of the Qwen3.6 model family, optimized explicitly for local repository-level agentic coding on Apple Silicon using the oMLX inference engine.
🎯 Why This Was Created
Standard open-weight releases and community pre-converted MLX files (like standard Unsloth) routinely strip out Multi-Token Prediction (mtp.*) tensors during conversion. While this saves minimal disk space, it completely disables the hardware-level Native MTP decoding pipeline on Apple Silicon.
This project was built to solve specific long-context constraints encountered on Apple Silicon:
- High-Throughput MoE: The Qwen3.6-35B-A3B is a sparse Mixture of Experts (MoE) model. It offers ultra-high throughput text generation and efficient inference while retaining a deep knowledge base.
- Overcoming the 128k Context Wall: Traditional backends like LM Studio frequently experience severe latency degradation or choke entirely when context windows approach 128k tokens due to unoptimized KV cache regeneration. Moving to oMLX's specialized two-tier caching eliminates this overhead.
- Balanced Performance & Precision: While oQ8 requires a high-end machine (like 64GB/96GB RAM), oQ5 (5-bit) offers a sweet spot of improved perplexity and adherence to complex code instructions over 4-bit quants, running smoothly on Macs with 36GB or 48GB Unified Memory.
🚀 Key Differences
| Feature / Attribute | Standard Qwen3.6 / Unsloth | This Custom Build (oQ5-fp16-mtp) |
|---|
| Native MTP Heads | Stripped out during conversion | Fully Preserved (Enables Native MTP toggle) |
| Vision Model (VLM) | Included by default (wastes memory space) | Stripped (Text/Coding only; ~2-3% smaller) |
| Quantization Method | Standard Uniform / GGUF Dynamic | oQ5 (Dynamic mixed-precision calibration) |
| Non-Quant Weight DType | bfloat16 | float16 (~20% faster prefill on M1/M2 Silicon) |
💻 Hardware & RAM Recommendations
| Mac Hardware Configuration | RAM Recommendation | Status / Performance Expectation |
|---|
| M1 / M2 / M3 Pro | 36GB / 48GB Unified Memory | Recommended (Optimal) — Runs smoothly with ample overhead for deep context windows. |
| M1 / M2 / M3 / M4 Max | 32GB / 64GB / 96GB | Fully Supported — High-speed bandwidth enables peak tokens/sec. (M2 Max 96GB will run this effortlessly). |
🛠️ Quantization Settings
This model was quantized using oMLX v0.4.4 with the following settings:
- Source Model:
Qwen/Qwen3.6-35B-A3B
- Sensitivity Model:
None (use source model)
- oQ Level:
oQ5
- Text Only:
Enabled (Excludes vision encoder weights, saving ~2-3% memory footprint)
- Preserve MTP weights:
Enabled (Preserves mtp.* tensors for Native MTP support)
- Non-quant weight dtype:
float16 (Provides ~20% faster prefill on M1/M2 Apple Silicon)
⚙️ Optimized oMLX Settings (v0.4.4)
To replicate the ultra-long context development workspace used with OpenCode 1.16.2, apply the following server specifications in your oMLX dashboard:
Model Basic Settings
- Reasoning Parser:
qwen_3_coder (Isolates CoT thoughts to protect IDE syntax)
- CTX Window:
262,144
- Max Tokens:
32,768
- Temperature:
0.6
- Top P / Top K:
0.95 / 20
- Min P:
0
- Repetition / Presence Penalty:
1 / 0
Model Advanced Settings
- Enabled Thinking: Checked (True)
- Chat Template Kwargs:
enable_thinking: true, preserve_thinking: true
- Native MTP: Checked (True)
Resource Management & Cache
- Memory Guard:
Aggressive (Enforces strict macOS memory/swap cleanup)
- Hot Cache Limit (RAM):
28GB (Allocated for high-speed Unified Memory history)
- Cold Cache Limit (SSD):
371GB (Serialized safetensors storage for context overflow)
- Max Concurrent Requests:
2 (Prevents splitting the 400 GB/s bandwidth bus)
- Embedding Batch Size:
32
- Chunked Prefill: Enabled (Prevents instantaneous out-of-memory crashes on large files)
- Burst Decode:
Aggressive (Coalesces tokens for maximum raw typing speed)
- Initial Cache Blocks:
256
- SSE Keepalive Mode:
Chunk
🌡️ Thermal Optimization Notice
Sustained local execution across deep context windows will heavily tax the Apple Silicon GPU/CPU, leading to internal heat buildup. Because Apple's default fan profiles prioritize silence over longevity, they delay high-RPM cooling until thermal throttling is already occurring.
To manage this proactively during extended coding agent sessions, use smc-command—an adapted CLI fan utility tailored to trigger more aggressive fan curves based on exact thermal sensors.
