Nemotron 3 Nano is a hybrid Mamba-2 + attention + MoE model (model_type: nemotron_h): 30B total parameters, ~3B active per token, 52 layers (only 6 of them attention), 128 experts with 6 active. Reasoning/thinking mode is enabled by the bundled chat template.

Conversion recipe

mlx_lm.convert \
  --hf-path nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 \
  -q --q-bits 4 --q-group-size 64

mlx-lm supports the nemotron_h architecture out of the box — no patches, no remote code.

Usage

bash

pip install -U mlx mlx-lm
mlx_lm.generate --model BrendanL79/Nemotron-3-Nano-30B-A3B-MLX-4bit \
  --prompt "Explain the difference between a hash table and a B-tree." \
  --max-tokens 512

The chat template enables a </think>-delimited reasoning preamble by default. Give it token budget (≥512) — at small budgets it spends the whole allowance thinking.

Measured on a Mac Mini M4 Pro 24GB (macOS, MLX)

metric	value
weights on disk	17 GB
peak inference memory	~17.1 GB — flat across a 1024-token generation
decode throughput	~95 tok/s sustained (−2% drift over 1024 tokens)
prefill	~620 tok/s @ 1K-token prompt (TTFT ≈ 1.6 s)
teacher-forced perplexity, public-domain prose, 1024-token window	2.98
chat-mode distinct-trigram ratio (1024 tokens, greedy)	0.962
raw-prompt (no template) behavior	robust — coherent continuation, 0.922 trigram

Perplexity caveat: the 2.98 figure is harness-specific. transformers' nemotron_h implementation scores the unquantized bf16 original ~5.7 on identical text and method (verified at both bf16 and fp32 on an H200-class GPU) — the two implementations diverge ~2x on this hybrid Mamba architecture for teacher-forced scoring. Compare perplexities within one backend only; the Qwen3.6 comparison below is mlx-lm vs mlx-lm and therefore sound.

The flat memory curve is the Mamba dividend: only 6 of 52 layers accumulate KV cache; the rest carry constant-size recurrent state. Transformer MoEs of comparable size grow hundreds of MB over the same generation. If you work at long context on a memory-constrained Mac, that property is the headline.

For comparison on the same machine and harness: Qwen3.6-35B-A3B-4bit does ~92 tok/s at 18.7 GB (growing), with stronger raw-prose perplexity (1.45). Pick Qwen3.6 for prose quality; pick this for long-context memory behavior and built-in reasoning mode.

Quantization notes

Grouped affine quantization, 4 bits, group size 64 (pinned explicitly — the mlx-community house recipe). Quality checks performed post-quant: coherent chat output with healthy trigram diversity, raw-mode continuation intact, perplexity in family-expected range. No mixed-precision recipe was needed; the MoE router layers survived the default treatment.

License & attribution

The base model is released by NVIDIA under the OpenMDW-1.1 license (Linux Foundation), which permits redistribution and derivative works including quantized variants. All credit for the model itself to NVIDIA's Nemotron team; this repo is a format conversion, nothing more. Quantized with mlx-lm.