Views
No views yet

| Benchmark | Nemotron-3-Nano-30B-A3B | Nemotron-3-Super-120B-A12B | Qwen3.5-35B-A3B | Nemotron-Cascade-2-30B-A3B |
|---|---|---|---|---|
| Math | ||||
| IMO 2025 | - | - | - | 🏅 35 pts |
| IMO AnswerBench | 70.4‡ | 77.2‡ | 74.8‡ | 79.3 |
| IMO ProofBench | - | - | - | 72.9 |
| AIME 2025 | 89.1 | 90.2 | 91.9‡ | 92.4 (98.6)† |
| AIME 2026 | 89.9‡ | 89.8‡ | 91.1‡ | 90.9 (95.0)† |
| HMMT Feb25 | 84.6‡ | 93.7 | 89.0 | 94.6 |
| Code Reasoning | ||||
| IOI 2025 | - | - | 348.6‡ | 🏅 439.3 |
| ICPC World Finals 2025 | - | - | - | 🏅 10/12 |
| LiveCodeBench v6 (2408-2505) | 68.3 | 78.7 | 74.6 | 87.2 (88.4)† |
| LiveCodeBenchPro 25Q2 (Easy) | 54.5‡ | 81.7‡ | 81.1‡ | 87.0 (89.3)† |
| LiveCodeBenchPro 25Q2 (Med) | 3.50‡ | 23.2‡ | 17.8‡ | 27.6 (36.8)† |
| SciCode | 33.3 | 42.1 | 38.0 | 36.4 |
| Knowledge & STEM | ||||
| MMLU-Redux | - | - | 93.3 | 86.3 |
| MMLU-Pro | 78.3 | 83.7 | 85.3 | 79.8 |
| GPQA-Diamond | 73.0 | 79.2 | 84.2 | 76.1 |
| HLE (no tool) | 10.6 | 18.3 | 22.4 | 17.7 |
| Alignment & Instruction Following | ||||
| ArenaHard v2 (Avg.) | 67.7 | - | 65.4‡ | 83.5 |
| – Hard Prompt | 72.1 | 73.9 | 64.5‡ | 88.2 |
| – Creative Writing | 63.2 | - | 66.3‡ | 78.7 |
| IFBench (prompt) | 71.5 | 72.6 | 70.2 | 82.9 |
| Scale AI Multi-Challenge | 38.5 | 55.2 | 60.0 | 45.3 |
| Long Context & Context Learning | ||||
| AA-LCR | 35.9 | 58.3 | 58.5 | 39.1 |
| LongBench v2 | 39.6 | - | 59.0 | 40.3 |
| NIAH@1M (RULER Subset) | 94.8 | 98.3 | 94.3‡ | 99.0 |
| CL-Bench | 12.0‡ | - | 15.5‡ | 12.2 |
| Agentic | ||||
| BFCL v4 | 53.8 | - | 67.3 | 52.9 |
| 𝜏²-Bench | 49.0 | 61.2 | 81.2 | 58.9 |
| Terminal Bench 2.0 | 8.5 | 31.0 | 40.5 | 21.1 |
| SWE Verified (OpenHands) | 38.8 | 60.5 | 69.2 | 50.2 |
| Multilingual | ||||
| MMLU-ProX | 59.5 | 79.4 | 81.0 | 72.5 |
| WMT24++ (en -> xx) | 86.2 | 86.7 | 87.6‡ | 84.1 |
<think> and </think> tags. To activate the instruct (non-thinking) mode, we prepend <think></think> to the beginning of the assistant’s response.tool role for tool responses; instead, we place them under the user role and warp them with <tool_response> and </tool_response>.http://localhost:8000/v1:vllm serve nvidia/Nemotron-Cascade-2-30B-A3B --port 8000 --tensor-parallel-size 1 --gpu-memory-utilization 0.9 --max-model-len 262144 --reasoning-parser nemotron_v3 --mamba-ssm-cache-dtype float32 --port 8000 --trust_remote_codevllm serve nvidia/Nemotron-Cascade-2-30B-A3B --port 8000 --tensor-parallel-size 1 --gpu-memory-utilization 0.9 --max-model-len 262144 --reasoning-parser nemotron_v3 --mamba-ssm-cache-dtype float32 --port 8000 --trust_remote_code --enable-auto-tool-choice --tool-call-parser qwen3_coder1from transformers import AutoTokenizer
2
3model_name = 'nvidia/Nemotron-Cascade-2-30B-A3B'
4tokenizer = AutoTokenizer.from_pretrained(model_name)
5
6'''
7single-turn example
8'''
9messages = [
10 {"role": "system", "content": "You are a helpful and harmless assistant.\n\nYou are not allowed to use any tools"},
11 {"role": "user", "content": "calculate 1+1?"}
12]
13
14# thinking mode
15prompt_thinking = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
16# prompt_thinking = '<|im_start|>system\nYou are a helpful and harmless assistant.\n\nYou are not allowed to use any tools<|im_end|>\n<|im_start|>user\ncalculate 1+1?<|im_end|>\n<|im_start|>assistant\n<think>\n'
17
18# instruct mode
19prompt_instruct = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
20# prompt_instruct = '<|im_start|>system\nYou are a helpful and harmless assistant.\n\nYou are not allowed to use any tools<|im_end|>\n<|im_start|>user\ncalculate 1+1?<|im_end|>\n<|im_start|>assistant\n<think></think>'
21
22'''
23multi-turn example
24'''
25messages = [
26 {"role": "system", "content": "You are a helpful and harmless assistant.\n\nYou are not allowed to use any tools"},
27 {"role": "user", "content": "calculate 1+1?"},
28 {"role": "assistant", "content": "<think>THINKING_CONTENT</think>\nTo calculate \\(1 + 1\\):\n\n1. **Identify the operation**: This is a basic addition problem involving two integers.\n2. **Perform the addition**: \n \\(1 + 1 = 2\\).\n\n**Result**: \\(\\boxed{2}\\)",},
29 {"role": "user", "content": "what about 2+2"}
30]
31
32# thinking mode
33prompt_thinking = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
34# prompt_thinking = '<|im_start|>system\nYou are a helpful and harmless assistant.\n\nYou are not allowed to use any tools<|im_end|>\n<|im_start|>user\ncalculate 1+1?<|im_end|>\n<|im_start|>assistant\n<think></think>\nTo calculate \\(1 + 1\\):\n\n1. **Identify the operation**: This is a basic addition problem involving two integers.\n2. **Perform the addition**: \n \\(1 + 1 = 2\\).\n\n**Result**: \\(\\boxed{2}\\)<|im_end|>\n<|im_start|>user\nwhat about 2+2<|im_end|>\n<|im_start|>assistant\n<think>\n'
35
36# instruct mode
37prompt_instruct = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
38# prompt_instruct = '<|im_start|>system\nYou are a helpful and harmless assistant.\n\nYou are not allowed to use any tools<|im_end|>\n<|im_start|>user\ncalculate 1+1?<|im_end|>\n<|im_start|>assistant\n<think></think>\nTo calculate \\(1 + 1\\):\n\n1. **Identify the operation**: This is a basic addition problem involving two integers.\n2. **Perform the addition**: \n \\(1 + 1 = 2\\).\n\n**Result**: \\(\\boxed{2}\\)<|im_end|>\n<|im_start|>user\nwhat about 2+2<|im_end|>\n<|im_start|>assistant\n<think></think>'1model_name = 'nvidia/Nemotron-Cascade-2-30B-A3B'
2tokenizer = AutoTokenizer.from_pretrained(model_name)
3
4SYSTEM_PROMPT = """# Tools
5
6You have access to the following functions:
7
8<tools>
9<function>
10<name>stateful_python_code_exec</name>
11<description>Call this function to execute Python code in a stateful Jupyter notebook environment. Python will respond with the output of the execution or time out after 120.0 seconds.</description>
12<parameters>
13<parameter>
14<name>code</name>
15<type>string</type>
16<description>Code to execute</description>
17</parameter>
18<required>["code"]</required>
19</parameters>
20</function>
21</tools>
22
23If you choose to call a function ONLY reply in the following format with NO suffix:
24
25<tool_call>
26<function=example_function_name>
27<parameter=example_parameter_1>
28value_1
29</parameter>
30<parameter=example_parameter_2>
31This is the value for the second parameter
32that can span
33multiple lines
34</parameter>
35</function>
36</tool_call>
37
38<IMPORTANT>
39Reminder:
40- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags
41- Required parameters MUST be specified
42- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after
43- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls
44</IMPORTANT>"""
45
46messages = [
47 {"role": "system", "content": SYSTEM_PROMPT},
48 {"role": "user", "content": "Solve the following math problem. Put your answer inside \\boxed{}.\n\nIn a school with 2008 students, each student is a member of certain committees. Each committee has at most 1004 members, and every two students are in at least one common committee. Determine the smallest possible number of committees in the school."}
49]
50
51prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
52print(prompt)1model_name = 'nvidia/Nemotron-Cascade-2-30B-A3B'
2tokenizer = AutoTokenizer.from_pretrained(model_name)
3
4SYSTEM_PROMPT = """You are a customer service agent that helps the user. The policy that determines how you should respond to requests from users is described below between the <policy> and </policy> tags.
5
6In each turn you can either:
7- Send a message to the user.
8- Make a tool call.
9You cannot do both at the same time.
10
11<policy>
12_NEED_TO_ADD_POLICY_HERE_
13</policy>
14
15Try to be helpful and always follow the policy.
16
17# Tools
18
19You have access to the following functions:
20
21<tools>
22<function>
23<name>_NEED_TO_ADD_FUNCTION_NAME_1_</name>
24<description>_FUNCTION_DESCRIPTION_</description>
25<parameters>
26<parameter>
27<name>_NEED_TO_ADD_PARAMETER_NAME_1_</name>
28<type>_PARAMETER_TYPE_</type>
29<description>_PARAMETER_DESCRIPTION_</description>
30<title>_PARAMETER_TITLE_</title>
31</parameter>
32<parameter>
33<name>_NEED_TO_ADD_PARAMETER_NAME_2_</name>
34<type>_PARAMETER_TYPE_</type>
35<description>_PARAMETER_DESCRIPTION_</description>
36<title>_PARAMETER_TITLE_</title>
37</parameter>
38...... (_MORE_PARAMETERS_TO_ADD_)
39<parameters>
40</function>
41...... (_MORE_FUNCTIONS_TO_ADD_)
42</tools>
43"""
44
45messages = [
46 {"role": "system", "content": SYSTEM_PROMPT},
47 {"role": "user", "content": "Hello, I'm calling regarding my upcoming stay at your hotel. My guest ID is G90920 and booking ID is B11246 for a Deluxe room on June 5th. I'm traveling with three 6-month-old triplets and need to request three infant cribs for our room. It's currently 30 hours before check-in—could you please confirm if this is feasible and if there are quiet room options available for families with infants?"}
48]
49
50prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
51print(prompt)@article{Nemotron_Cascade_2,
title={Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation},
author={Yang, Zhuolin and Liu, Zihan and Chen, Yang and Dai, Wenliang and Wang, Boxin and Lin, Sheng-Chieh and Lee, Chankyu and Chen, Yangyi and Jiang, Dongfu and He, Jiafan and Pi, Renjie and Lam, Grace and Lee, Nayeon and Bukharin, Alexander and Shoeybi, Mohammad and Catanzaro, Bryan and Ping, Wei},
year={2026}
}