Qwen-AgentWorld-35B-A3B-oQ3.5
oQ3.5 (data-driven mixed-precision, ≈3.5 bpw) MLX quantization of
Qwen/Qwen-AgentWorld-35B-A3B,
produced with
oMLX's
quantize_oq_streaming.
For Apple Silicon. Runs in
mlx-lm, oMLX, or any MLX app. Siblings:
oQ4
(higher-quality, ≈4.6 bpw) and
bf16 (full precision).
Notes
- Text-only. The base checkpoint declares a
vision_config and MTP heads in
config.json, but ships no vision or mtp.* weights (693 tensors, 0 vision,
0 MTP) — both are vestigial skeleton inherited from the Qwen3.5 base. This quant
is the faithful language model; nothing multimodal was dropped.
- ≈16 GB on disk (from ≈69 GB bf16). Peak memory ≈17.6 GB generating.
- Mixed-precision: per-layer bit allocation from oQ's sensitivity measurement;
most weights are 4-bit-class with sensitive layers boosted.
Performance
Measured with
oMLX (Auto engine) on an
M5 Max (40-core GPU, 128 GB RAM). Single request:
| Context (pp/tg) | TTFT | decode | prefill | peak mem |
|---|
| 1024 / 128 | 461 ms | 148 tok/s | 2223 tok/s | 17.3 GB |
| 4096 / 128 | 1.17 s | 139 tok/s | 3510 tok/s | 18.1 GB |
| 8192 / 128 | 2.21 s | 135 tok/s | 3707 tok/s | 18.4 GB |
| 32768 / 128 | 11.5 s | 118 tok/s | 2856 tok/s | 20.4 GB |
Continuous batching (pp1024/tg128): 1×→148 · 2×→193 · 4×→265 · 8×→352 tok/s
aggregate decode (2.37× at 8 concurrent requests).
Reference: BF16 source
Same setup, full-precision
Qwen/Qwen-AgentWorld-35B-A3B:
| Context (pp/tg) | TTFT | decode | prefill | peak mem |
|---|
| 1024 / 128 | 644 ms | 77 tok/s | 1591 tok/s | 65.6 GB |
| 4096 / 128 | 1.68 s | 76 tok/s | 2434 tok/s | 66.4 GB |
| 8192 / 128 | 2.39 s | 75 tok/s | 3428 tok/s | 66.7 GB |
| 32768 / 128 | 12.0 s | 67 tok/s | 2730 tok/s | 68.7 GB |
Continuous batching (pp1024/tg128): 1×→77 · 2×→68 · 4×→114 · 8×→119 tok/s.
Takeaway: oQ3.5 gives ≈1.9× the single-request decode throughput at ≈¼ the
memory (17 GB vs 66 GB), and scales better under batching (2.37× vs 1.55× at 8×).
Accuracy (quick reference)
A quick, non-representative sanity check — 100-question samples per
benchmark with thinking enabled, run via oMLX's accuracy bench. Not enough
to draw firm conclusions, but it gives a rough idea of how much quality the
quant retains vs the BF16 source.
| Benchmark | BF16 | oQ4 | oQ3.5 |
|---|
| MathQA | 85.0% | 84.0% | 83.0% |
| MMLU-Pro | 76.0% | 77.0% | 72.0% |
oQ4 tracks BF16 within ≈1 pp; oQ3.5 trades a few more points (notably on
MMLU-Pro) for smaller size and faster decode.
Usage
1mlx_lm.generate --model mlx-community/Qwen-AgentWorld-35B-A3B-oQ3.5 \
2 --system-prompt "You are a language world model simulating a Linux terminal. Given the user's command, predict the terminal output." \
3 --prompt $'Action: execute_bash\nCommand: ls -la /home/user/project/' \
4 --max-tokens 512 --temp 0.6
The model uses thinking mode (
<think>...</think>) by default. Recommended
sampling:
temperature=0.6,
top_p=0.95,
top_k=20. See the
base model card for the
seven agent domains and domain-specific system prompts.