MLX 3-bit quantization of InternScience/Agents-A1 (affine, group size 64). The source is bf16; this is a uniform mlx quantization.

Agents-A1 is a Qwen3.5-MoE vision-language agent model (qwen3_5_moe, Qwen3_5MoeForConditionalGeneration): 40 decoder layers, 256 routed experts per layer + a shared expert, hidden size 2048, with a vision tower and video preprocessing.

Multimodal (VLM) — load with mlx-vlm (mlx-lm can't load multimodal architectures):

Loads and runs in stock mlx-vlm — no patched code needed at inference.

I first tried oMLX's data-driven oQ quantization, but it doesn't work for this checkpoint: oQ writes the MoE experts in a per-expert layout that omlx's own loader can't read back (parameters not in model), so the quantized model fails to load. This build therefore uses standard mlx-vlm quantization instead — uniform 3-bit, group size 64 — which loads cleanly in both stock mlx-vlm & oMLX.

Measured with oMLX's benchmark harness on a Macbook Pro M5 Max 128GB 40 GPU — gen 128 tokens, cold prefill (unique prompt prefix per request, no cache reuse).

TTFT (cold prefill) is ~precision-independent — ≈0.3 s @1k, 3 s @8k, 21 s @32k, 63 s @64k, ~225 s @128k — prefill is compute-bound, not weight-bound.

Aggregate across the batch; per-request rate is that value divided by the batch size.

17 x 24 -> correct (408), coherent, no repetition.

apache-2.0, inherited from the base model.