NVFP4 (4-bit) quantized build of Nex-N2-mini — an agentic model with Agentic Thinking.
This repository contains a post-training-quantized checkpoint of nex-agi/Nex-N2-mini, produced with llm-compressor and stored in the compressed-tensors format. The MoE expert stack is quantized to NVFP4 (W4A4) and the attention / output projections to FP8 (W8A8), shrinking the checkpoint from 70.2 GB to 24.8 GB (~2.8×) so the model fits comfortably on a single GPU.
Applied to attention, the linear-attention projections, the LM head, and the MoE stack of the last 8 layers (32–39), which are the most quantization-sensitive:
self_attn.{q,k,v,o}_proj
linear_attn.{in_proj_qkv,in_proj_z,out_proj}
lm_head
layers.{32..39}.mlp.experts.*.{gate,up,down}_proj and layers.{32..39}.mlp.shared_expert.{gate,up,down}_proj
Weights
Activations
Format
FP8 (E4M3)
FP8 (E4M3)
Strategy
per-channel, symmetric
per-token, symmetric
Calibration
static, memoryless_minmax
dynamic
Kept in BF16
The full vision tower (model.visual.*, 333 tensors)
MoE routers — mlp.gate, mlp.shared_expert_gate
Linear-attention gating / state projections — linear_attn.in_proj_a, linear_attn.in_proj_b, linear_attn.conv1d, A_log, dt_bias, and all linear_attn.norm*
Token embeddings and all RMSNorm weights
The KV cache is not quantized (kv_cache_scheme: null).
Hardware
NVFP4 reaches its full speedup on NVIDIA Blackwell GPUs (SM100 / SM120 — B200, GB200, RTX PRO 6000 Blackwell), which have native FP4 tensor cores. vLLM can still load and run this checkpoint on pre-Blackwell hardware through its NVFP4 dequantization/emulation path, in which case you get the memory savings but not the throughput gain.
At 24.8 GB of weights the model serves on a single 80 GB card with room for a long-context KV cache; the launch command below is tuned for that shape.
Usage
Deployment with vLLM
Note:Qwen3.5-MoE support, the NVFP4 compressed-tensors kernels, and the --language-model-only flag all require a recent vLLM build. Use a current release or nightly.
Nex-series models emit explicit reasoning traces and support robust function calling. To have vLLM split the reasoning trace from the final answer and parse tool calls into the OpenAI tool_calls schema, add:
Without these flags the reasoning trace and the <tool_call> blocks are returned inline in content, and you parse them yourself.
Recommended Sampling Parameters
temperature: 0.7
top_p: 0.95
top_k: 40
Deployment with SGLang
The base model is best served with the customized sglang fork described in the Nex-N2-mini card. That fork does not currently consume compressed-tensors NVFP4 checkpoints — use the BF16 base model there, and this repository with vLLM.
Performance
The scores below are the published results for the BF16 base models and are reproduced here for reference. They were not re-measured on this NVFP4 checkpoint; expect a small degradation typical of 4-bit post-training quantization. If you benchmark it, we would welcome the numbers in the Community tab.
Benchmark
Nex-N2-mini
Nex-N2-Pro
GPT-5.5
Opus 4.7
Kimi-K2.6
GLM-5.1
MiniMax M3
DeepSeek-V4-Pro
Agent
BrowseComp
74.1
83.7
84.4
79.8
83.2
79.3
83.5
83.4
GDPval
1402
1585
1769
1753
1481
1535
-
1554
Toolathlon
33.3
51.9
55.6
52.8
50.0
40.7
-
51.8
WildClawBench
47.7
53.5
58.2
62.2
-
48.2
-
43.7
WideSearch
62.0
75.6
-
-
80.8
-
-
-
TAU3
65.9
71.1
-
-
-
70.6
-
-
Coding & SWE
SWE-Bench Pro
50.2
58.8
58.6
64.3
58.6
58.4
59.0
55.4
Terminal-Bench 2.1
60.7
75.3
83.4
69.7
-
58.7
66.0
72.0
DeepSWE
8.0
33.6
70
54
24
18
-
8
SWE-Bench Verified
74.4
80.8
82.9
87.6
80.2
-
80.5
80.6
SWE Atlas QnA
31.5
37.9
45.4
45.2
-
-
37.9
-
SWE Atlas RF
30.0
32.9
44.8
48.6
-
-
-
-
SWE Atlas TW
23.3
40.0
42.6
38.2
-
-
30.8
-
General & Reasoning
GPQA Diamond
82.6
90.7
93.6
94.2
90.5
86.2
-
90.1
IFEval
89.1
94.0
-
-
94.5
94.5
-
91.9
Apex
9.4
36.5
-
-
24.0
11.5
-
38.3
Nex-N2 Benchmark Overview
About Nex-N2
Nex-N2 is an agent model built for real-world productivity scenarios. With first-tier coding and agentic capabilities, it keeps driving complex, long-horizon tasks forward in real environments to deliver stable, end-to-end results.
Rather than treating reasoning, tool use, and environment execution as separate capabilities, Nex-N2 unifies them through an Agentic Thinking framework that connects requirement understanding, task planning, code implementation, environmental feedback, evaluation and debugging, and continuous iteration into a single closed loop. The framework has two parts:
Adaptive Thinking lets the model decide on its own when to think and how deeply — executing simple actions quickly while reasoning thoroughly on critical decisions.
Coherent Thinking carries one consistent reasoning paradigm across general reasoning and diverse agentic tasks, staying consistent across tasks and modalities to enable stable capability transfer.
Nex-N2 ships in two variants, both post-trained on the Qwen3.5 series: Nex-N2-Pro (built on Qwen3.5-397B-A17B) and Nex-N2-mini (built on Qwen3.5-35B-A3B-Base).
Model Architecture
Architecture
Qwen3_5MoeForConditionalGeneration (hybrid linear + full attention MoE)
Layers
40 (full attention every 4th layer, linear attention elsewhere)
