The pretraining data has a cutoff date of September 2024.
Model Overview
NVIDIA Nemotron-Elastic-12B is a large language model (LLM) developed by NVIDIA for research purposes. This model uses a hybrid architecture consisting primarily of Mamba-2 and MLP layers combined with just four Attention layers, designed to enable elastic inference through nested model extraction.
The model was post-trained from NVIDIA-Nemotron-Nano-12B-v2, incorporating advanced reasoning capabilities and optimized for mathematical and scientific reasoning tasks.
A key innovation of this model is its Elastic Architecture, which enables the extraction of smaller, nested variants (6B and 9B parameters) from the same parameter space without requiring separate training runs. For more details, please refer to the full paper.
This model is for research and development only.
The figure below illustrates the overall training and deployment pipelines of Nemotron-Elastic.
Elastic Overview
This approach provides significant advantages over traditional model compression methods:
Cost Efficiency Benefits
Training Token Savings: Nemotron Elastic achieves a 7.2× reduction in training tokens compared to traditional compression methods. While approaches like Minitron require separate exploratory and knowledge distillation phases for each target size (750B tokens for 6B+9B variants), Nemotron Elastic trains all variants simultaneously in a single run requiring only 110B tokens.
Deployment Memory Efficiency: The nested weight-sharing architecture provides substantial memory advantages. Deploying all three model variants (6B, 9B, and 12B) requires only 24GB memory - equivalent to storing just the 12B model alone. This represents a 42% memory reduction compared to storing separate 9B and 12B checkpoints (42GB), while providing an additional 6B variant at no extra cost.
Configuration
Models
Total Memory (BF16)
Nemotron Elastic
6B + 9B + 12B
24 GB
NanoV2
9B + 12B
42 GB
Scalable Architecture: Unlike traditional compression methods (such as Minitron-SSM, which was used to create the 9B variant from the 12B NanoV2 model) where costs scale linearly with the number of target model sizes, Nemotron Elastic maintains approximately constant training and memory overhead regardless of how many nested variants are extracted. This scalability makes it particularly valuable for edge deployment scenarios that require multiple model sizes to handle varying workloads or user-selected quality-latency tradeoffs.
Number of Parameters: 12B Elastic (encapsulates 6B and 9B)
Deployment Geography: Global
Use Case: This model is intended for researchers studying elastic inference, hybrid architectures, mathematical reasoning, and AI systems that require flexible computational resource allocation.
The Nemotron-Elastic-12B model supports extraction of smaller nested variants:
Nemotron-Elastic-6B: 6B parameter variant extracted from the 12B model
Nemotron-Elastic-9B: 9B parameter variant extracted from the 12B model
These variants are extracted using the provided slicing script at slice_nemotron_elastic.py.
Input
Input Type(s): Text
Input Format(s): String
Input Parameters: One-Dimensional (1D): Sequences
Other Properties Related to Input: Context length up to 128K. Supported languages include English and multilingual capabilities.
Output
Output Type(s): Text
Output Format: String
Output Parameters: One-Dimensional (1D): Sequences up to 128K
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
Model Version
v1.0
Prompt Format
We follow the jinja chat template provided below. This template conditionally adds <think>\n to the start of the Assistant response if /think is found in either the system prompt or any user message. If no reasoning signal is added, the model defaults to reasoning "on" mode. The chat template adds <think></think> to the start of the Assistant response if /no_think is found in the system prompt. Thus enforcing reasoning on/off behavior.
{%- set ns = namespace(enable_thinking = true) %}
{%- for message in messages -%}
{%- set content = message['content'] -%}
{%- if message['role'] == 'user' or message['role'] == 'system' -%}
{%- if '/think' in content -%}
{%- set ns.enable_thinking = true -%}
{%- elif '/no_think' in content -%}
{%- set ns.enable_thinking = false -%}
{%- endif -%}
{%- endif -%}
{%- endfor -%}
{%- if messages[0]['role'] != 'system' -%}
{%- set ns.non_tool_system_content = '' -%}
{{- '<SPECIAL_10>System\n' -}}
{%- else -%}
{%- set ns.non_tool_system_content = messages[0]['content']
.replace('/think', '')
.replace('/no_think', '')
.strip()
-%}
{{- '<SPECIAL_10>System\n' + ns.non_tool_system_content }}
{%- endif -%}
{%- if tools -%}
{%- if ns.non_tool_system_content is defined and ns.non_tool_system_content != '' -%}
{{- '\n\n' -}}
{%- endif -%}
{{- 'You can use the following tools to assist the user if required:' -}}
{{- '\n<AVAILABLE_TOOLS>[' -}}
{%- for tool in tools -%}
{{- (tool.function if tool.function is defined else tool) | tojson -}}
{{- ', ' if not loop.last else '' -}}
{%- endfor -%}
{{- ']</AVAILABLE_TOOLS>\n\n' -}}
{{- 'If you decide to call any tool(s), use the following format:\n' -}}
{{- '<TOOLCALL>[{{"name": "tool_name1", "arguments": "tool_args1"}}, ' -}}
{{- '{{"name": "tool_name2", "arguments": "tool_args2"}}]</TOOLCALL>\n\n' -}}
{{- 'The user will execute tool-calls and return responses from tool(s) in this format:\n' -}}
{{- '<TOOL_RESPONSE>[{{"tool_response1"}}, {{"tool_response2"}}]</TOOL_RESPONSE>\n\n' -}}
{{- 'Based on the tool responses, you can call additional tools if needed, correct tool calls if any errors are found, or just respond to the user.' -}}
{%- endif -%}
{{- '\n' -}}
{%- set messages = messages[1:] if messages[0]['role'] == 'system' else messages -%}
{%- if messages[-1]['role'] == 'assistant' -%}
{%- set ns.last_turn_assistant_content = messages[-1]['content'].strip() -%}
{%- set messages = messages[:-1] -%}
{%- endif -%}
{%- for message in messages -%}
{%- set content = message['content'] -%}
{%- if message['role'] == 'user' -%}
{{- '<SPECIAL_11>User\n' + content.replace('/think', '').replace('/no_think', '').strip() + '\n' }}
{%- elif message['role'] == 'tool' -%}
{%- if loop.first or (messages[loop.index0 - 1].role != 'tool') -%}
{{- '<SPECIAL_11>User\n' + '<TOOL_RESPONSE>[' }}
{%- endif -%}
{{- message['content'] -}}
{{- ', ' if not loop.last and (messages[loop.index0 + 1].role == 'tool') else '' -}}
{%- if loop.last or (messages[loop.index0 + 1].role != 'tool') -%}
{{- ']</TOOL_RESPONSE>\n' -}}
{%- endif -%}
{%- elif message['role'] == 'assistant' -%}
{%- if '</think>' in content -%}
{%- set content = content.split('</think>')[1].strip() %}
{%- endif -%}
{{- '<SPECIAL_11>Assistant\n' + content.strip() }}
{%- if message.tool_calls -%}
{%- if content.strip() != '' -%}
{{- '\n\n' -}}
{%- endif -%}
{{- '<TOOLCALL>[' -}}
{%- for call in message.tool_calls -%}
{%- set fn = call.function if call.function is defined else call -%}
{{- '{"name": "' + fn.name + '", "arguments": ' -}}
{%- if fn.arguments is string -%}
{{- fn.arguments -}}
{%- else -%}
{{- fn.arguments | tojson -}}
{%- endif -%}
{{- '}' + (', ' if not loop.last else '') -}}
{%- endfor -%}
{{- ']</TOOLCALL>' -}}
{%- endif -%}
{{- '\n<SPECIAL_12>\n' -}}
{%- endif -%}
{%- endfor -%}
{%- if add_generation_prompt -%}
{{- '<SPECIAL_11>Assistant\n' -}}
{%- if ns.enable_thinking is defined and ns.enable_thinking is false -%}
{{- '<think></think>' -}}
{%- else -%}
{{- '<think>\n' -}}
{%- endif -%}
{%- if ns.last_turn_assistant_content is defined and ns.last_turn_assistant_content != '' -%}
{{- ns.last_turn_assistant_content -}}
{%- endif -%}
{%- else -%}
{%- if ns.last_turn_assistant_content is defined and ns.last_turn_assistant_content != '' -%}
{{- '<SPECIAL_11>Assistant\n' -}}
{%- if ns.enable_thinking is defined and ns.enable_thinking is false -%}
{{- '<think></think>' -}}
{%- else -%}
{{- '<think>\n' -}}
{%- endif -%}
{{- ns.last_turn_assistant_content -}}
{%- if continue_final_message is defined -%}
{%- if continue_final_message is false -%}
{{- '\n<SPECIAL_12>\n' -}}
{%- endif -%}
{%- else -%}
{{- '\n<SPECIAL_12>\n' -}}
{%- endif -%}
{%- endif -%}
{%- endif -%}
Training, Testing, and Evaluation Datasets
Training datasets
Data Modality: Text
Text Training Data Size: More than 10 Trillion Tokens
Train/Test/Valid Split: We used 100% of the corpus for pre-training and relied on external benchmarks for testing.
Data Collection Method by dataset: Hybrid: Automated, Human, Synthetic
Labeling Method by dataset: Hybrid: Automated, Human, Synthetic
Properties: The post-training corpus for NVIDIA-Nemotron-Nano-12B-v2 consists of English and multilingual text (German, Spanish, French, Italian, Korean, Portuguese, Russian, Japanese, Chinese and English). Our sources cover a variety of document types such as: webpages, dialogue, articles, and other written materials. The corpus spans domains including code, legal, math, science, finance, and more. We also include a small portion of question-answering, and alignment style data to improve model accuracies. For several of the domains listed above we used synthetic data, specifically reasoning traces, from DeepSeek R1/R1-0528, Qwen3-235B-A22B, Nemotron 4 340B, Qwen2.5-32B-Instruct-AWQ, Qwen2.5-14B-Instruct, Qwen 2.5 72B.
The pre-training corpus for NVIDIA-Nemotron-Nano-12B-v2 consists of high-quality curated and synthetically-generated data. It is trained in the English language, as well as 15 multilingual languages and 43 programming languages. Our sources cover a variety of document types such as: webpages, dialogue, articles, and other written materials. The corpus spans domains including legal, math, science, finance, and more. We also include a small portion of question-answering, and alignment style data to improve model accuracy. The model was pre-trained for approximately twenty trillion tokens.
Alongside the model, we release our final pretraining data, as outlined in this section. For ease of analysis, there is a sample set that is ungated. For all remaining code, math and multilingual data, gating and approval is required, and the dataset is permissively licensed for model training purposes.
Private Non-publicly Accessible Datasets of Third Parties
Dataset
Global Regulation
Workbench
Online Dataset Sources
The English Common Crawl data was downloaded from the Common Crawl Foundation (see their FAQ for details on their crawling) and includes the snapshots CC-MAIN-2013-20 through CC-MAIN-2025-13. The data was subsequently deduplicated and filtered in various ways described in the Nemotron-CC paper.
Additionally, we extracted data for fifteen languages from the following three Common Crawl snapshots: CC-MAIN-2024-51, CC-MAIN-2025-08, CC-MAIN-2025-18. The fifteen languages included were Arabic, Chinese, Danish, Dutch, French, German, Italian, Japanese, Korean, Polish, Portuguese, Russian, Spanish, Swedish, and Thai. As we did not have reliable multilingual model-based quality classifiers available, we applied just heuristic filtering instead—similar to what we did for lower quality English data in the Nemotron-CC pipeline, but selectively removing some filters for some languages that did not work well. Deduplication was done in the same way as for Nemotron-CC.
The GitHub Crawl was collected using the GitHub REST API and the Amazon S3 API. Each crawl was operated in accordance with the rate limits set by its respective source, either GitHub or S3. We collect raw source code and subsequently remove any having a license which does not exist in our permissive-license set (for additional details, refer to the technical report).
Data Collection Method by dataset: Hybrid: Human, Synthetic
Labeling Method by dataset: Hybrid: Automated, Human, Synthetic
Benchmark Results
Reasoning Evaluations (Reasoning ON)
The following table shows performance across key reasoning and mathematical benchmarks. All Nemotron-Elastic variants and Nanov2 baselines represent checkpoints at the end of 49k context distillation run (prior to RL and checkpoint merging). Other models represents the final, public version.
Elastic Model Accuracy
The accuracy shown is the average across all benchmarks: MATH-500, AIME-2024, AIME-2025, GPQA, LiveCodeBench v5, and MMLU-Pro.
Benchmark Descriptions:
MATH-500: A subset of 500 questions from the MATH benchmark testing mathematical problem-solving capabilities.
AIME-2024/2025: American Invitational Mathematics Examination problems testing advanced mathematical reasoning.
The slicing process preserves the hybrid architecture while reducing model size through structured pruning of embedding dimensions and MLP layers.
Potential Known Risks for Usage
The model was trained on data that contains toxic language, unsafe content, and societal biases originally crawled from the internet. Therefore, the model may amplify those biases and return toxic responses especially when prompted with toxic prompts. The model may generate answers that may be inaccurate, omit key information, or include irrelevant or redundant text producing socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive. Code produced by the model may not always model real-world contexts and should be checked. The model demonstrates weakness to alignment-breaking attacks. Users are advised to deploy language model guardrails alongside this model to prevent potentially harmful outputs.
Ethical Considerations
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
Please report security vulnerabilities or NVIDIA AI Concerns here.
Research Applications
This model is particularly suitable for research in:
Elastic Inference: Studying adaptive model sizing based on computational constraints
Hybrid Architectures: Exploring the combination of Mamba-2 and Transformer layers
Model Compression: Understanding structured pruning and nested model extraction
Resource-Adaptive AI: Developing systems that can scale computational requirements dynamically
Citation
bibtex
1@article{taghibakhshi2025nemotron,
2 title={Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs},
3 author={Taghibakhshi, Ali and Sreenivas, Sharath Turuvekere and Muralidharan, Saurav and Cai, Ruisi and Chochowski, Marcin and Mahabaleshwarkar, Ameya Sunil and Suhara, Yoshi and Olabiyi, Oluwatobi and Korzekwa, Daniel and Patwary, Mostofa and others},
4 journal={arXiv preprint arXiv:2511.16664},
5 year={2025}
6}
Example Usage
python
1import torch
2from transformers import AutoTokenizer, AutoModelForCausalLM
34# Load the tokenizer and model (full 12B version)5tokenizer = AutoTokenizer.from_pretrained("nvidia/Nemotron-Elastic-12B", trust_remote_code=True)6model = AutoModelForCausalLM.from_pretrained("nvidia/Nemotron-Elastic-12B", torch_dtype=torch.bfloat16, trust_remote_code=True).cuda()78# Use the prompt template9messages =[10{"role":"system","content":"You are a helpful mathematical reasoning assistant"},11{"role":"user","content":"Solve the following equation: 2x + 5 = 15"},12]13tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to(model.device)1415outputs = model.generate(tokenized_chat, max_new_tokens=512)16print(tokenizer.decode(outputs[0]))
Note: This example uses the full 12B model directly. Alternatively, you can extract smaller variants (6B or 9B) using the slicing script mentioned above if you need reduced computational requirements for your specific deployment scenario.
