Puzzle 75B A9B has 75.3B total parameters and 9.3B active parameters per token. A uniform conversion is not the best fit for a 64 GB Mac because most parameters live in the routed expert bank, while dense paths and the output head are more sensitive.

This checkpoint applies precision by role:

Component	Precision
Routed MoE expert projections	4-bit affine
Dense projections and shared experts	6-bit affine
Token embeddings	6-bit affine
Output head	BF16
Quantization group size	64

The ordinary autoregressive backbone is included. Training-only MTP tensors were intentionally excluded.

Measured M2 Max results

Tested on an Apple M2 Max with 64 GB unified memory using MLX 0.32.0.

Measurement	Result
Directory size	42.03 GiB
Peak MLX memory	49.6835 GB
Maximum process RSS	45.3008 GB
Average generation speed	14.2722 tok/s
Local task checks	24/30
Long-context retrieval checks	4/5

A separate 256-token Python-generation run measured 24.623 tok/s and 45.269 GB peak process memory. That run used a different prompt and should not be compared directly with the 30-run average.

The local checks are regression and instruction-following checks, not a standardized model-quality benchmark. See the benchmark methodology and raw metrics.

MLX support

This model needs the Nemotron Puzzle support implemented in:

Until the upstream pull request is merged and released, install the feature branch:

bash

git clone --branch feat/nemotron-h-puzzle-support \
  https://github.com/sxuff/mlx-lm.git
cd mlx-lm
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .

Generate

bash

mlx_lm.generate \
  --model tekosML/Nemotron-Labs-3-Puzzle-75B-A9B-MLX-4bit-experts-6bit \
  --trust-remote-code \
  --prompt "Write a Python function that returns the first n Fibonacci numbers." \
  --max-tokens 256 \
  --temp 1.0 \
  --top-p 0.95

Only use --trust-remote-code after inspecting the repository's configuration and modeling files. They were copied from the NVIDIA source checkpoint during conversion.

Conversion

The conversion script and exact heterogeneous quantization predicate are available at:

Equivalent conversion command:

bash

python scripts/convert_puzzle_mixed.py \
  /path/to/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16 \
  /path/to/output \
  --expert-bits 4

Recommended sampling settings used during validation:

text

temperature = 1.0
top_p = 0.95

Numerical compatibility work

The supporting MLX implementation preserves Puzzle's BF16 softplus(dt + dt_bias) boundary while retaining FP32 state-transition and recurrence arithmetic. First-layer SSM cosine similarity against NVIDIA's reference improved from 0.8832 to 0.999998 after correcting that boundary.

Puzzle-specific RMSNorm, router, projection-bias, and timestep behavior are gated to nemotron_h_puzzle, preserving existing ordinary Nemotron-H behavior.

Files and provenance

Source weights: nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16
Conversion runtime: MLX 0.32.0
Format: sharded safetensors
Weight shards: 9
Total local files: 20
File-content size: 45,104,700,954 bytes

License

Use and redistribution are governed by the original model's OpenMDW-1.1 license. The full license text and NVIDIA copyright notice are included in this repository.

The files configuration_nemotron_h.py and modeling_nemotron_h.py are separately licensed under Apache-2.0 as stated in the included license.

Original model copyright:

Users are responsible for reviewing and complying with the original model license.