You can deploy this model and start earning money today!

pipenetwork/
NVIDIA-Nemotron-3-Nano-4B-MLX-8bit

Name: NVIDIA-Nemotron-3-Nano-4B-MLX-8bit
Availability: InStock

Views

No views yet

Other quantizations: 4-bit · 8-bit (this)

📚 Part of the Nemotron-3 MLX (Apple Silicon) collection.

library_name: mlx license: other license_name: nvidia-nemotron-open-model-license license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/ pipeline_tag: text-generation language:

en tags:
nvidia
pytorch
mlx base_model: nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 datasets:
nvidia/Nemotron-CC-v2
nvidia/Nemotron-Post-Training-Dataset-v2
nvidia/Nemotron-Science-v1
nvidia/Nemotron-Instruction-Following-Chat-v1
nvidia/Nemotron-Agentic-v1
nvidia/Nemotron-Competitive-Programming-v1
nvidia/Nemotron-Math-Proofs-v1
nvidia/Nemotron-RL-Agentic-Conversational-Tool-Use-Pivot-v1
nvidia/Nemotron-RL-instruction_following
nvidia/Nemotron-RL-agent-calendar_scheduling
nvidia/Nemotron-RL-instruction_following-structured_outputs track_downloads: true

MLX build notes (this repo)

8-bit MLX quantization of NVIDIA's Nemotron-3 Nano 4B (dense hybrid: 42 layers = 21 Mamba-2 + 17 MLP + 4 Attention), converted from the BF16 release for Apple Silicon.


Precision	MLX affine 8-bit, group 64 (~8.5 bpw)
Size	3.9 GB
Runs on	Apple Silicon, ~16 GB+ unified memory

Near-lossless and still tiny — recommended default for the 4B.

Usage

Requires trust_remote_code. The 4B uses MLP layers that the built-in transformers nemotron_h config (5.5.x) does not yet support, so it ships its own config code.

mlx_lm.server --model pipenetwork/NVIDIA-Nemotron-3-Nano-4B-MLX-8bit --trust-remote-code

python

from mlx_lm import load, generate
model, tok = load(
    "pipenetwork/NVIDIA-Nemotron-3-Nano-4B-MLX-8bit",
    tokenizer_config={"trust_remote_code": True},
)

Fixes applied in this repo

EOS fix: upstream generation_config.json had eos_token_id: 2, but the chat template ends turns with <|im_end|> (id 11); without 11 in the stop set, generation never halts. This repo sets eos_token_id: [2, 11].
Reasoning model: emits <think>...</think> before the final answer.

Conversion

mlx_lm.convert --hf-path nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 \
  --mlx-path <out> -q --q-bits 8 --q-group-size 64 --trust-remote-code

Converted by pipenetwork on Apple Silicon (mlx-lm 0.31.2). Not affiliated with NVIDIA; license and attribution per the original model card above.