NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16

Looking for the post-trained model? See NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 for the full-precision release, or NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 for optimized inference.

Model Overview

• The pre-training data has a cutoff date of September 2025.

Description

What is Nemotron?

License/Terms of Use

Benchmarks

Deployment Geography: Global

Use Case

Release Date

Reference(s)

• NVIDIA Nemotron model family on Hugging Face

Model Architecture

• Architecture Type: Mamba2-Transformer Hybrid Mixture of Experts (MoE) with Multi-Token Prediction (MTP)
• Network Architecture: Nemotron Hybrid MoE
• Number of model parameters: 30B Total / 3B Active

Model Design

Training Methodology

• NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16 model was pre-trained using an NVFP4 recipe with crawled and synthetic code, math, science, and general knowledge data.
• Software used for pre-training: Megatron-LM

• The model underwent a continued pre-training phase to train its Multi-Token Prediction (MTP) layers. In this stage, MTP heads learn to predict multiple future tokens, providing richer training signals to the base model and enabling native speculative decoding at inference time.

Input

• Input Type(s): Text
• Input Format(s): String
• Input Parameters: One-Dimensional (1D): Sequences
• Other Properties Related to Input: Maximum context length up to 1M tokens. The pre-training corpus spans English, 19 other spoken languages, and 43 programming languages.

Output

• Output Type(s): Text
• Output Format: String
• Output Parameters: One-Dimensional (1D): Sequences
• Other Properties Related to Output: Maximum context length up to 1M tokens

Software Integration

• Runtime Engine(s): PyTorch; Megatron-LM; NeMo
• Supported Hardware Microarchitecture Compatibility: NVIDIA Ampere - A100; NVIDIA Blackwell; NVIDIA Hopper - H100-80GB
• Operating System(s): Linux

Model Version(s)

• v1.0 - GA (08/11/2026)

Training, Testing, and Evaluation Datasets:

Training

Testing Datasets:

• Hybrid: Automated, Human, Synthetic Labeling Method by dataset
  
• Hybrid: Automated, Human, Synthetic Properties: This corpus comprises a mix of high-quality standard benchmarks for base language models, covering general knowledge, math, code, commonsense understanding, reading comprehension, multilingual understanding, and long context.

Evaluation Datasets:

• Hybrid: Automated, Human, Synthetic Labeling Method by dataset
  
• Hybrid: Automated, Human, Synthetic Properties: This corpus comprises a mix of high-quality standard benchmarks for base language models, covering general knowledge, math, code, commonsense understanding, reading comprehension, multilingual understanding, and long context.

Inference

• Acceleration Engine: PyTorch
• Test Hardware:
  • NVIDIA Hopper
    • 1-8x H100
    • 1-8x H200
  • NVIDIA Blackwell
    • GB200

Ethical Considerations