MLX 2bit conversion of thinkingmachines/Inkling-Small for Apple Silicon (mlx-vlm). 276B total / 12B active sparse MoE (42 layers, 256 routed experts top-6 + 2 shared), text + image + audio in, text out.

Requires mlx-vlm >= 0.6.9

0.6.9 is the first release whose models/inkling can load an official Inkling checkpoint through the public loader, and the first that implements the MoE global_scale / gate.bias tensors. On 0.6.7 / 0.6.8 this repo will not load.

python

from mlx_vlm import load, generate
model, processor = load("ToPo-ToPo/Inkling-Small-mlx-2bit")

The config is the official schema, unmodified — no key translation and no loader patches are needed.

Provenance (self-converted from official weights)

Source: thinkingmachines/Inkling-Small (license: apache-2.0, bf16, 531.9 GB)
Tool: mlx-vlm 0.6.9 — mlx_vlm.convert --hf-path thinkingmachines/Inkling-Small --mlx-path . -q --q-bits 2 --q-group-size 64
Effective: 2.506 bits/weight (77 GiB on disk)
Only edit on top of the conversion: pad_token / eos_token added to tokenizer_config.json (the official TokenizersBackend config sets neither, so transformers raises on any padded call). Both point at existing ids — the vocabulary is unchanged.

Reasoning effort

The chat template always injects a Thinking effort level: system message (default 0.9). Control it with the OpenAI-compatible reasoning_effort — "none" / "minimal" / "low" / "medium" / "high" / "max", or a float in [0.0, 0.99]. "none" disables thinking entirely.

Revision history

2026-08-04 — reconverted with mlx-vlm 0.6.9. The previous upload had been converted with 0.6.7, whose models/inkling did not implement the MoE mlp.global_scale (50 keys) and mlp.gate.bias (40 keys) present in the official checkpoint, so those tensors were silently dropped. It also shipped a translated config (renamed intermediate_size / dense_intermediate_size, etc.) that 0.6.9 rejects. If you pulled this repo before this date, re-download it.