Views
No views yet
| Task | Metric | Nemotron-3-Ultra 550B-A55B-Base | DeepSeek-V3.2 Exp-Base | Mistral-Large-3 675B-Base-2512 | Kimi-K2 Base | GLM-4.5 Base |
|---|---|---|---|---|---|---|
| General Knowledge | ||||||
| MMLU | 5-shot, acc | 89.08 | 87.82 | 87.35 | 87.60 | 86.50 |
| MMLU-Pro | 5-shot, CoT EM | 79.07 | 63.26 | 67.42 | 69.15 | 65.78 |
| AGIEval-En | 3/5-shot, CoT EM | 78.73 | 70.13 | 69.30 | 72.55 | 70.06 |
| GPQA | 5-shot, CoT EM | 50.00 | 31.82 | 34.85 | 43.43 | 34.85 |
| Math | ||||||
| GSM8K | 8-shot, CoT EM | 88.10 | 84.38 | 91.21 | 91.05 | 85.37 |
| MATH | 4-shot, EM | 82.00 | 60.12 | 62.88 | 68.40 | 57.58 |
| Code | ||||||
| HumanEval | sampled pass@1 n=32, EvalPlus sanitized | 83.84 | 61.85 | 66.71 | 78.20 | 78.16 |
| MBPP-Sanitized | 3-shot pass@1 n=32, EvalPlus sanitized | 85.97 | 58.66 | 84.08 | 72.14 | 76.69 |
| Commonsense Understanding | ||||||
| ARC-Challenge | 25-shot, acc_norm | 97.35 | 95.22 | 97.27 | 95.82 | 96.59 |
| HellaSwag | 10-shot, acc_norm | 90.51 | 89.44 | 88.88 | 90.92 | 90.17 |
| OpenBookQA | 0-shot, acc_norm | 48.60 | 48.20 | 51.40 | 50.80 | 49.60 |
| PIQA | 0-shot, acc_norm | 83.79 | 85.09 | 84.82 | 85.47 | 85.09 |
| WinoGrande | 5-shot, acc | 79.32 | 83.43 | 82.08 | 84.21 | 85.24 |
| Reading Comprehension | ||||||
| RACE | 0-shot, acc | 92.15 | 93.21 | 93.30 | 91.96 | 92.15 |
| Multilingual | ||||||
| MMLU Global Lite | 5-shot, avg | 90.13 | 85.59 | 87.34 | 85.63 | 85.81 |
| MGSM | 8-shot, native CoT avg | 87.73 | 82.33 | 82.93 | 85.20 | 81.27 |
| Long Context | ||||||
| RULER 64K | 0-shot | 95.30 | 93.30 | 90.11 | 93.79 | 16.12 |
| RULER 128K | 0-shot | 92.49 | 91.88 | 55.77 | 88.61 | 0.00 |
| RULER 256K | 0-shot | 86.22 | -- | 35.50 | -- | -- |
| RULER 512K | 0-shot | 84.54 | -- | -- | -- | -- |
| RULER 1M | 0-shot | 76.83 | -- | -- | -- | -- |
| Dataset Collection | Token Counts | Description |
|---|---|---|
| Nemotron-CC-v2 & v2.1 | 9.1T | A massive collection of English web data filtered from Common Crawl, including 2.5T+ tokens of new organic, translated, and synthetically rephrased content. |
| Nemotron-CC-Code-v1 | 427.9B | High-quality code tokens extracted from Common Crawl using the Lynx + LLM pipeline to preserve structure and equations. |
| Nemotron-Pretraining-Code-v1 & v2 & v3 | 1.7T | Curated GitHub code references with multi-stage filtering, deduplication, and large-scale synthetic code data. |
| Nemotron-CC-Math-v1 | 133.3B | High-quality math pre-training dataset preserving LaTeX formatting and mathematical structures. |
| Nemotron-Pretraining-Specialized-v1 & v1.1 & v1.2 & Nemotron-Pretraining-SFT-v1 | 660.0B | Synthetic datasets targeting specialized domains such as STEM reasoning and scientific coding. |
| Nemotron-Pretraining-Legal-v1 | 4.3B | Synthetic datasets targeting the legal domain. |
| Dataset | Modality | Dataset Size | Collection Period | Collecting Organisation |
|---|---|---|---|---|
| English Common Crawl | Text | 3.36T | 4/8/2025 | NVIDIA Advanced Deep Learning Research |
| English Common Crawl 1.1 | Text | Not disclosed | 10/2/2025 | NVIDIA Advanced Deep Learning Research |
| Multilingual Common Crawl | Text | 812.7B | 5/1/2025 | NVIDIA Advanced Deep Learning Research |
| GitHub Crawl | Text | 747.4B | 4/29/2025 | NVIDIA Advanced Deep Learning Research |
| GitHub Crawl 1.1 | Text | 172.7B | 9/30/2025 | NVIDIA Advanced Deep Learning Research |
| Dataset | Model(s) used |
|---|---|
| Global Regulation | Unknown |
| TAUS Translation Memory | Unknown |
| Scale HLE | Unknown |
| HackerRank Coding | Unknown |
| RL data for Search | Gemini 3; GPT-5 |
| Dataset | Model(s) used |
|---|---|
| Simple Minesweeper | - |
| Simple Sudoku | - |
| Multitool Typewriter Hard | - |
| Machine Translation of News Commentary and TAUS Translation Memory | - |
| Machine Translation of STEM - | Qwen2.5-14B-Instruct |
| Competitive Coding RL data from Nemotron Cascade | - |
| Long context RL | - |
| Single-step SWE RL for patch generation | - |
| OpenHands SWE | - |
| Dataset | Modality | Dataset Size | Seed Dataset | Model(s) used for generation |
|---|---|---|---|---|
| Nemotron-Pretraining-Fact-Seeking | Text | 35.0B | FineWiki | Qwen3-30B-A3B-Instruct-2507 |
| Nemotron-Pretraining-Legal | Text | 4.3B | CommonPile (caselaw_access_project_filtered); California Code of Regulations; Judicial Ethics Opinions; GLOBALCIT; CUAD; Nemotron Personas; ToSDR Terms of Service Corpus; CodeHima/TOS_Dataset; ContractNLI; CaseHOLD; Code of Federal Regulations; Canadian Case Law (subsets that allow commercial use) | Qwen3-235B-A22B-Thinking-2507 |
| Nemotron-Pretraining-Formal-Logic | Text | 128M | Nemotron Personas | Qwen3-235B-A22B-Thinking-2507 |
| Nemotron-Pretraining-Economics | Text | 73.4M | - | Qwen3-235B-A22B-Thinking-2507 |
| Nemotron-Pretraining-Multiple-Choice | Text | 1.6B | MMLU Auxiliary Train | DeepSeek-V3; Qwen3-235B-A22B |
| Nemotron-Pretraining-Code-Concepts | Text | 7.3B | - | gpt-oss-20b; gpt-oss-120b |
| Nemotron-Pretraining-Unconditional-Algorithmic | Text | 196.5M | - | gpt-oss-120b; Qwen3-235B-A22B |
| More Synthetic Tasks from DeepSeek-V3 and Qwen3-235B-A22B | Text | 1.1B | train splits of acp_bench; ai2_arc; babi; gsm8k; hendrycks_math; IFEval; MedText; mediqa_qa; mlqa; MMLU-Pro; mmlu-pro-plus; MMLU-ProX; nq_open; tinyGSM8k; truthful_qa; truthfulqa-multi; MATH-lighteval; mmlu; awesome-chatgpt-prompts; super_glue | DeepSeek v3; Qwen3-235B-A22B |
| Synthetic Tasks from DeepSeek-V3 and Qwen3-235B-A22B | Text | 6.7B | train splits of Into the Unknown; AI2 ARC (AI2 Reasoning Challenge); BLiMP (Benchmark of Linguistic Minimal Pairs); CommonSenseQA; GLUE; HeadQA; Hendrycks Ethics; Memo Trap; modus-tollens; NeQA; pattern-matching-suppression; mastermind_24_mcq_random; mastermind_24_mcq_close; quote-repetition; redefine-math; Repetitive Algebra; sig-figs; MMLU-Pro; MC-TACO; MedConceptsQA; MMLU_dataset; OpenbooksQA; PIQA (Physical Interaction Question Answering); SocialIQA; SuperGLUE; tinyAI2_arc; tinyMMLU; tinyWinogrande; TruthfulQA; WebQuestions; Winogrande; GPQA; MBPP | DeepSeek v3; Qwen3-235B-A22B |
| Synthetic Art of Problem Solving from DeepSeek-R1 | Text | 40B | Art of Problem Solving; American Mathematics Competitions 8; American Mathematics Competitions 10; | DeepSeek-R1 |
| Synthetic Moral Stories and Social Chemistry from Qwen3-235B-A22B-Thinking-2507 and Mixtral-8x22B-v0.1 | Text | 15.2M | social-chemestry-101; Moral Stories | Qwen3-235B-A22B-Thinking-2507; Mixtral-8x22B-v0.1 |
| Synthetic Moral Stories and Social Chemistry from Mixtral-8x22B-v0.1 | Text | 327M | social-chemestry-101; Moral Stories | Mixtral-8x22B-v0.1 |
| Synthetic Social Sciences seeded with OpenStax from DeepSeek-V3, Mixtral-8x22B-v0.1, and Qwen2.5-72B | Text | 83.6M | OpenStax - CC BY-SA subset | DeepSeek-V3; Mixtral-8x22B-v0.1; Qwen2.5-72B |
| Synthetic Health Sciences seeded with OpenStax from DeepSeek-V3, Mixtral-8x22B-v0.1, and Qwen2.5-72B | Text | 9.7M | OpenStax - CC BY-SA subset | DeepSeek-V3; Mixtral-8x22B-v0.1; Qwen2.5-72B |
| Synthetic STEM seeded with OpenStax, Open Textbook Library, and GSM8K from DeepSeek-R1, DeepSeek-V3, DeepSeek-V3-0324, and Qwen2.5-72B | Text | 175M | OpenStax - CC BY-SA subset; GSM8K; Open Textbook Library - CC BY-SA & GNU subset | DeepSeek-R1, DeepSeek-V3; DeepSeek-V3-0324; Qwen2.5-72B |
| Nemotron-PrismMath | Text | 4.6B | Big-Math-RL-Verified; OpenR1-Math-220k | Qwen2.5-0.5B-instruct, Qwen2.5-72B-Instruct; DeepSeek-R1-Distill-Qwen-32B |
| Synthetic Question Answering Data from Papers and Permissible Books from Qwen2.5-72B-Instruct | Text | 350M | arXiv; National Institutes of Health ExPorter; BioRxiv; PMC Article; USPTO Backgrounds; peS2o; Global Regulation; CORE; PG-19; DOAB CC BY & CC BY-SA subset; NDLTD | Qwen2.5-72B-Instruct |
| Synthetic Rephrased Math Data from Common Crawl from phi-4 | Text | 73B | Common Crawl | phi-4 |
| Synthetic Math Data from Common Crawl 4plus | Text | 52.3B | Common Crawl | phi-4 |
| Synthetic Math Data from Common Crawl 3 | Text | 80.9B | Common Crawl | phi-4 |
| Synthetic AGIEval seeded with AQUA-RAT, LogiQA, and AR-LSAT from DeepSeek-V3 and DeepSeek-V3-0324 | Text | 4.0B | AQUA-RAT; LogiQA; AR-LSAT | DeepSeek-V3; DeepSeek-V3-0324 |
| Synthetic AGIEval seeded with AQUA-RAT, LogiQA, and AR-LSAT from Qwen3-30B-A3B | Text | 4.2B | AQUA-RAT; LogiQA; AR-LSAT | Qwen3-30B-A3B |
| Synthetic Art of Problem Solving from Qwen2.5-32B-Instruct, Qwen2.5-Math-72B, Qwen2.5-Math-7B, and Qwen2.5-72B-Instruct | Text | Art of Problem Solving; American Mathematics Competitions 8; American Mathematics Competitions 10; GSM8K; PRM800K | Qwen2.5-32B-Instruct; Qwen2.5-Math-72B; Qwen2.5-Math-7B; Qwen2.5-72B-Instruct | |
| Synthetic MMLU Auxiliary Train from DeepSeek-R1 | Text | 0.5B | MMLU Auxiliary Train | DeepSeek-R1 |
| Synthetic Long Context Continued Post-Training Data from Papers and Permissible Books from Qwen2.5-72B-Instruct | Text | arXiv; National Institutes of Health ExPorter; BioRxiv; PMC Article; USPTO Backgrounds; peS2o; Global Regulation; CORE; PG-19; DOAB CC BY & CC BY-SA subset; NDLTD | Qwen2.5-72B-Instruct | |
| Synthetic Common Crawl from Qwen3-30B-A3B and Mistral-Nemo-12B-Instruct | Text | 415.8B | Common Crawl | Qwen3-30B-A3B; Mistral-NeMo-12B-Instruct |
| Synthetic Multilingual Data from Common Crawl from Qwen3-30B-A3B | Text | Common Crawl | Qwen3-30B-A3B | |
| Synthetic Multilingual Data from Wikimedia from Qwen3-30B-A3B | Text | Wikimedia | Qwen3-30B-A3B | |
| Synthetic Math Data from Wikimedia from Nemotron-4-340B-Instruct | Text | - | Nemotron-4-340B-Instruct | |
| Synthetic Common Crawl Code from phi-4 | Text | 427.9B | Common Crawl | phi-4 |
| Synthetic Scientific Coding from Qwen3-235B-A22B | Text | 1.2B | Wikimedia | Qwen3-235B-A22B |
| Tool Calling Data | Text | 26.2B | Qwen3-235B-A22B-2507; gpt-oss-120b | |
| Synthetic Essential-Web from QwQ-32B | Text | 28.1B | Essential-Web | QwQ-32B |
| Translated Synthetic Crawl | Text | 389.9B | Common Crawl | Qwen3-30B-A3B |
| Translated Synthetic Wikipedia | Text | 7.9B | Wikimedia | Qwen3-30B-A3B |
| Synthetic Long Context from Qwen3-235B-A22B-Instruct-2507 | Text | Undisclosed | CORE; PG-19; DOAB CC BY & CC BY-SA subset; NDLTD | Qwen3-235B-A22B-Instruct-2507 |
| Synthetic Search STEM OPENQ from DeepSeek-R1-0528 | Text | Undisclosed | - | DeepSeek-R1-0528 |
| Synthetic MCQ from Qwen2.5-32B-Instruct and DeepSeek-R1-0528 | Text | Undisclosed | - | Qwen2.5-32B-Instruct; DeepSeek-R1-0528 |
| Synthetic Offline Search MCQA HLE from DeepSeek-R1-0528 | Text | Undisclosed | - | DeepSeek-R1-0528 |
| Synthetic Offline Search MCQA GPQA from Qwen3-235B-A22B and DeepSeek-R1-0528 | Text | Undisclosed | - | Qwen3-235B-A22B; DeepSeek-R1-0528 |
| Synthetic Human Preference from QwQ-32B, Qwen3-30B-A3B, Qwen3-235B-A22B, Qwen3-235B-A22B-Instruct-2507, Mistral-Small-3.1-24B-Instruct-2503, Mistral-Small-3.2-24B-Instruct-2506, MiniMax-M1-80k, MiniMax-M1-40k, Kimi-K2-Instruct, DeepSeek-V3-0324, DeepSeek-R1-0528 | Text | Undisclosed | - | QwQ-32B; Qwen3-30B-A3B; Qwen3-235B-A22B; Qwen3-235B-A22B-Instruct-2507; Mistral-Small-3.1-24B-Instruct-2503; Mistral-Small-3.2-24B-Instruct-2506; MiniMax-M1-80k; MiniMax-M1-40k; Kimi-K2-Instruct; DeepSeek-V3-0324; DeepSeek-R1-0528 |
| Synthetic Code from Qwen3-32B | Text | Undisclosed | English Common Crawl; English Common Crawl 1.1 | Qwen3-32B |
| Synthetic OpenCodeReasoning from DeepSeek-R1 | Text | Undisclosed | OpenCodeReasoning | DeepSeek-R1 |
| Synthetic LIMO from DeepSeek-R1-0528 | Text | Undisclosed | LIMO | DeepSeek-R1-0528 |
| Synthetic SCP from DeepSeek-R1-0528 | Text | Undisclosed | SCP-116K | DeepSeek-R1-0528 |
| Synthetic Stack Exchange from DeepSeek-R1-0528 | Text | Undisclosed | Stack Exchange | DeepSeek-R1-0528 |
| Synthetic Common Crawl from Qwen3-30B-A3B | Text | Undisclosed | Common Crawl | Qwen3-30B-A3B |
| Synthetic Wikipedia from Qwen3-30B-A3B | Text | Undisclosed | Wikimedia | Qwen3-30B-A3B |
| Synthetic Essential-Web from Qwen3-30B-A3B and Qwen3-235B-A22B-Thinking-2507 | Text | Undisclosed | Essential-Web | Qwen3-30B-A3B; Qwen3-235B-A22B-Thinking-2507 |
| Synthetic Textbook Math from Qwen3-30B-A3B, Qwen3-235B-A22B, phi-4 | Text | Undisclosed | Common Crawl; FineMath | Qwen3-30B-A3B; Qwen3-235B-A22B; phi-4 |
| Synthetic Math and Code from DeepSeek-R1 and DeepSeek-R1-0528 | Text | Undisclosed | Magicoder-Evol-Instruct-110K; opc-sft-stage2; TACO; OpenCodeReasoning; OpenMathReasoning; NuminaMath CoT | DeepSeek-R1; DeepSeek-R1-0528 |
1@misc{nvidia_nemotron_3_ultra_2026,
2 title = {Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning},
3 author = {{NVIDIA}},
4 year = {2026},
5 url = {https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Ultra-Technical-Report.pdf},
6 note = {White Paper}
7}