Qwen3.6-35B-A3B oQ8 Text-Only with Native MTP (Optimized for Apple Silicon)
This repository contains custom-quantized, text-only configurations of the Qwen3.6 model family, optimized explicitly for local repository-level agentic coding on Apple Silicon using the oMLX inference engine.
🎯 Why This Was Created
Standard open-weight releases and community pre-converted MLX files (like standard Unsloth) routinely strip out Multi-Token Prediction (mtp.*) tensors during conversion. While this saves minimal disk space, it completely disables the hardware-level Native MTP decoding pipeline on Apple Silicon.
This project was built to solve specific long-context constraints encountered on a MacBook Pro M2 Max (96GB Unified Memory, 400 GB/s memory bandwidth):
- High-Throughput MoE: The Qwen3.6-35B-A3B is a sparse Mixture of Experts (MoE) model. It offers ultra-high throughput text generation and efficient inference while retaining a deep knowledge base.
- Overcoming the 128k Context Wall: Traditional backends like LM Studio frequently experience severe latency degradation or choke entirely when context windows approach 128k tokens due to unoptimized KV cache regeneration. Moving to oMLX's specialized two-tier caching eliminates this overhead.
- Maximized Precision: While 4-bit and 5-bit quants are popular for saving memory, a 96GB unified memory architecture provides plenty of overhead. These builds use oQ8 (8-bit) precision to guarantee lossless adherence to complex syntax, indentation, and nested tool calls.
🚀 Key Differences
| Feature / Attribute | Standard Qwen3.6 / Unsloth | This Custom Build (oQ8-fp16-mtp) |
|---|
| Native MTP Heads | Stripped out during conversion | Fully Preserved (Enables Native MTP toggle) |
| Vision Model (VLM) | Included by default (wastes memory space) | Stripped (Text/Coding only; ~2-3% smaller) |
| Quantization Method | Standard Uniform / GGUF Dynamic | oQ8 (Dynamic mixed-precision calibration) |
| Non-Quant Weight DType | bfloat16 | float16 (~20% faster prefill on M1/M2 Silicon) |
💻 Hardware & RAM Recommendations
| Mac Hardware Configuration | RAM Recommendation | Status / Performance Expectation |
|---|
| M1 Max / M2 Max / M3 Max (Base 32GB/36GB/48GB) | 48GB Unified Memory | Supported (Minimal) — Good for standard queries, but context window will be limited. |
| M1 / M2 / M3 / M4 Max / Ultra | 64GB Unified Memory | Recommended — Great performance, handles deep context windows (~128k) comfortably. |
| M2 Max / M3 Max (96GB / 128GB) | 96GB / 128GB Unified Memory | Optimal / Best Experience — The user's M2 Max 96GB is ideal. Allows running maximum context (262k) with zero slowdowns and plenty of system overhead. |
🛠️ Quantization Settings
This model was quantized using oMLX v0.4.3 with the following settings:
- Source Model:
Qwen/Qwen3.6-35B-A3B
- Sensitivity Model:
None (use source model)
- oQ Level:
oQ8
- Text Only:
Enabled (Excludes vision encoder weights, saving ~2-3% memory footprint)
- Preserve MTP weights:
Enabled (Preserves mtp.* tensors for Native MTP support)
- Non-quant weight dtype:
float16 (Provides ~20% faster prefill on M1/M2 Apple Silicon)
⚙️ Optimized oMLX Settings (v0.4.3)
To replicate the ultra-long context development workspace used with OpenCode 1.16.2, apply the following server specifications in your oMLX dashboard:
Model Basic Settings
- Reasoning Parser:
qwen_3_coder (Isolates CoT thoughts to protect IDE syntax)
- CTX Window:
262,144
- Max Tokens:
32,768
- Temperature:
0.6
- Top P / Top K:
0.95 / 20
- Min P:
0
- Repetition / Presence Penalty:
1 / 0
Model Advanced Settings
- Enabled Thinking: Checked (True)
- Chat Template Kwargs:
enable_thinking: true, preserve_thinking: true
- Native MTP: Checked (True)
Resource Management & Cache
- Memory Guard:
Aggressive (Enforces strict macOS memory/swap cleanup)
- Hot Cache Limit (RAM):
40GB (Allocated for high-speed Unified Memory history)
- Cold Cache Limit (SSD):
371GB (Serialized safetensors storage for context overflow)
- Max Concurrent Requests:
2 (Prevents splitting the 400 GB/s bandwidth bus)
- Embedding Batch Size:
32
- Chunked Prefill: Enabled (Prevents instantaneous out-of-memory crashes on large files)
- Burst Decode:
Aggressive (Coalesces tokens for maximum raw typing speed)
- Initial Cache Blocks:
256
- SSE Keepalive Mode:
Chunk
🌡️ Thermal Optimization Notice
Sustained local execution across deep context windows will heavily tax the Apple Silicon GPU/CPU, leading to internal heat buildup. Because Apple's default fan profiles prioritize silence over longevity, they delay high-RPM cooling until thermal throttling is already occurring.
To manage this proactively during extended coding agent sessions, use smc-command—an adapted CLI fan utility tailored to trigger more aggressive fan curves based on exact thermal sensors.
