Read our How to Run Nemotron 3 Nano Guide!

See Unsloth Dynamic 2.0 GGUFs for our quantization benchmarks.

NVIDIA-Nemotron-3-Nano-30B-A3B-FP8

Model Overview

• The post-training data has a cutoff date of November 28, 2025.
• The pre-training data has a cutoff date of June 25, 2025.

Description

What is Nemotron?

Feature Voting

License/Terms of Use

Reasoning Benchmark Evaluations

Deployment Geography: Global

Use Case

Release Date

Reference(s)

• NVIDIA Nemotron 3 model family on Hugging Face
• NVIDIA Nemotron 2 model family on Hugging Face
• NVIDIA Nemotron 3 White Paper

Model Architecture

• Architecture Type: Mamba2-Transformer Hybrid Mixture of Experts (MoE)
• Network Architecture: Nemotron Hybrid MoE
• Number of model parameters: 30B

Model Design

Training Methodology

• NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 model was pre-trained using crawled and synthetic code, math, science, and general knowledge data. All datasets are disclosed in the Training, Testing, and Evaluation Datasets section of this document. Major portions of the pre-training corpus are released in the Nemotron-Pre-Training-Datasets collection.
• Software used for pre-training: Megatron-LM

• The model was further fine-tuned on synthetic code, math, science, tool calling, instruction following, structured outputs, and general knowledge data. All datasets are disclosed in the Training, Testing, and Evaluation Datasets section of this document. Major portions of the fine-tuning corpus are released in the Nemotron-Post-Training-v3 collection.
• Software used for supervised fine-tuning: Megatron-LM

• The model underwent multi-environment reinforcement learning using synchronous GRPO (Group Relative Policy Optimization) across math, code, science, instruction following, multi-step tool use, multi-turn conversations, and structured output environments. Conversational quality was further refined through RLHF using a generative reward model. All datasets are disclosed in the Training, Testing, and Evaluation Datasets section of this document. The RL environments and datasets are released as part of NeMo Gym.
• Software used for reinforcement learning: NeMo RL, NeMo Gym

• The model including the KV cache was quantized into FP8. To preserve accuracy while improving efficiency, we applied a selective quantization strategy, keeping the attention layers and the Mamba layers that feed into those attention layers in BF16.
• Software used for quantization: Model Optimizer

Input

• Input Type(s): Text
• Input Format(s): String
• Input Parameters: One-Dimensional (1D): Sequences
• Maximum input size: 128K tokens
• Other Properties Related to Input: Supported languages include: English, Spanish, French, German, Japanese, Italian

Output

• Output Type(s): Text
• Output Format: String
• Output Parameters: One-Dimensional (1D): Sequences
• Maximum output size: 128K tokens

Software Integration

• Runtime Engine(s): NeMo 25.11.01
• Supported Hardware Microarchitecture Compatibility: NVIDIA H100-80GB, NVIDIA A100
• Operating System(s): Linux

Use it with Transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8")
model = AutoModelForCausalLM.from_pretrained(
    "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto"
)

messages = [
    {"role": "user", "content": "Write a haiku about GPUs"},
]

tokenized_chat = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    tokenized_chat,
    max_new_tokens=1024,
    temperature=1.0,
    top_p=1.0,
    eos_token_id=tokenizer.eos_token_id
)
print(tokenizer.decode(outputs[0]))

tokenized_chat = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    enable_thinking=False,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

# Use Greedy Search for reasoning off
outputs = model.generate(
    tokenized_chat,
    max_new_tokens=32,
    do_sample=False,
    num_beams=1,
    eos_token_id=tokenizer.eos_token_id
)
print(tokenizer.decode(outputs[0]))

Use it with vLLM

VLLM_USE_FLASHINFER_MOE_FP8=1 vllm serve  --model nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 \
 --max-num-seqs 8 \
  --tensor-parallel-size 1 \
  --max-model-len 262144 \
  --port 8000 \
  --trust-remote-code \
  --tool-call-parser qwen3_coder \
  --reasoning-parser-plugin nano_v3_reasoning_parser.py \
  --reasoning-parser nano_v3 \
  --kv-cache-dtype fp8

Use it with TRT-LLM

# nano_v3 example yaml is https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/auto_deploy/nano_v3.yaml
trtllm-serve <model_path> \
--backend _autodeploy \
--trust_remote_code \
--reasoning_parser nano-v3 \
--tool_parser qwen3_coder \
--extra_llm_api_options nano_v3.yaml

Use it with SGLang

For more detailed information on how to use the model with SGLang, please see this cookbook.

python3 -m sglang.launch_server --model-path nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 \
  --trust-remote-code \
  --tp 1 \
  --attention-backend flashinfer \
  --tool-call-parser qwen3_coder \
  --reasoning-parser nano_v3

Model Version(s)

• v1.0

Training, Testing, and Evaluation Datasets

Private Non-publicly Accessible Datasets of Third Parties

Private Non-publicly Accessible Datasets by NVIDIA

Crawled and Scraped from Online Sources by NVIDIA

NVIDIA-Sourced Synthetic Datasets

Training Dataset

Language Distribution in Post-Training

Evaluation Dataset

• Data Collection Method by dataset: Hybrid: Human, Synthetic
• Labeling Method by dataset: Hybrid: Automated, Human, Synthetic

Inference

• Engines: HF, vLLM, TRT-LLM, SGLang, Llama.cpp
• Test Hardware: NVIDIA A100 80GB, H100 80GB, B200 192GB, RTX PRO 6000 96GB

Ethical Considerations

Citation

@misc{nvidia_nemotron_nano_v3_2025,
  title  = {{Nemotron 3 Nano}: Open, Efficient Mixture-of-Experts Hybrid {Mamba}-{Transformer} Model for {Agentic} Reasoning},
  author = {{NVIDIA}},
  year   = {2025},
  url    = {https://research.nvidia.com/labs/adlr/files/NVIDIA-Nemotron-nano-v3-report.pdf},
  note   = {Technical report}
}