BF16 is ~44% slower per request due to higher memory bandwidth requirements (~65GB vs 19GB). Use 4-bit for throughput; use BF16 for maximum output quality on reasoning tasks.

Usage

pip install mlx-lm

python

from mlx_lm import load, generate

model, tokenizer = load("Vaynel/Qwen3.6-35B-A3B-BF16-MLX")
response = generate(model, tokenizer, prompt="Write a binary search in Python.", max_tokens=512)
print(response)

Model Details

Architecture: Qwen3MoeForCausalLM (MoE, 35B total / 3.6B active)
Context: 131,072 tokens
Precision: BF16 (bfloat16)
Size: ~65GB
Hardware tested: Apple M3 Ultra 512GB