A 120-billion-parameter model that runs on a 36 GB Mac. This is a 2-bit mixed-precision MLX quant of NVIDIA's Nemotron-3-Super-120B-A12B (247 GB at bf16), produced by mlx-optiq. It is 47.5 GB on disk. While it generates, only ~14 GB sits in RAM: the Mamba blocks, attention, router and shared experts stay resident, and the 34 GB of routed mixture-of-experts weights stream off the SSD as the router selects them, through optiq serve --stream-experts.

Nemotron-3-Super is a hybrid: Mamba2 state-space blocks interleaved with attention and a 512-expert sparse MoE (22 active per token). Asked to write Flappy Bird in a single HTML file, the 2-bit model produced a complete, working game. Here it is playing it:

For a model this large, exact calibration-driven sensitivity is impractical (it would run for days and needs the full model resident as a reference), so OptiQ's static method assigns bits from architecture alone. See the methods comparison.

This is a Nemotron hybrid (model_type: nemotron_h), so it needs mlx-lm from main and import optiq (install from git, not a version pin):

Serve it with SSD expert streaming (auto-enabled for a MoE too big to fit resident; --stream-experts forces it):

Then open the Lab, ask for a game, and watch it render in the Canvas pane. Only the routed experts stream per token; the Mamba state, attention and shared experts stay resident, so the footprint stays ~14 GB no matter how large the model on disk is.

This is an extreme quant. 2-bit on the routed experts is lossy, and the point of this artifact is that a 120 B hybrid MoE runs at all on consumer Apple Silicon, with coherent output. For reference quality on this base, use the bf16 weights or a higher-bit quant. The full story is in the blog post.