You can deploy this model and start earning money today!

chankhavu/
Nemotron-Cascade-2-30B-A3B-NVFP4

Name: Nemotron-Cascade-2-30B-A3B-NVFP4
Availability: InStock

Views

No views yet

Nemotron-Cascade-2-30B-A3B-NVFP4

NVFP4 (4-bit) quantization of Nemotron-Cascade-2-30B-A3B. Quantized with NVIDIA ModelOpt using the same selective recipe as NVIDIA's official Nano NVFP4: MoE experts and Mamba GEMMs in NVFP4 (E2M1 with block scaling), attention and sensitive layers in BF16, KV cache in FP8. Native FP4 compute on Blackwell; weight-only dequant on Hopper/Ampere.

Benchmarks

Calculated using NVIDIA-NeMo/Evaluator with config from Nemotron-3-Super-120B's eval config:

Benchmark	Nemotron-Cascade-2-30B-A3B (reproduced results)	Nemotron-Cascade-2-30B-A3B-NVFP4 (this model)
AIME 2025 (avg@8)	98.8	97.9
AIME 2026 (avg@8)	94.2	92.1
HMMT Feb 2025 (avg@8)	92.9	90.1

With the low sample count (8 rollouts per problem), a deviation of ±2% accross runs is expected. The NVFP4 is consistently 1-2% below the original BF16.

Quantization Details

Method: NVFP4 Post-Training Quantization (PTQ), without Quantization-Aware Distillation (QAD)
Format: E2M1 (1 sign, 2 exponent, 1 mantissa bit) with hierarchical block scaling
Block scaling: Group size 16 — each block of 16 values shares an FP8 E4M3 scale, plus a per-tensor FP32 global scale
KV cache: FP8
Tooling: NVIDIA ModelOpt

Selective Quantization Recipe

Follows the Nano-architecture selective quantization recipe from the Nemotron 3 Nano Technical Report (Section 4). Same recipe as NVIDIA's official NVFP4 checkpoint. Sensitive components are kept in higher precision:

Component	Precision	Rationale
MoE expert GEMMs (routed + shared)	NVFP4	All 23 MoE layers, 128 routed + 2 shared experts each
Mamba GEMMs (non-adjacent)	NVFP4	17 of 23 Mamba layers
Attention layers (all 6)	BF16	Most sensitive — kept BF16 per NVIDIA sensitivity analysis
Mamba layers adjacent to attention (6)	BF16	Layers {4, 11, 18, 25, 32, 41} — found sensitive in ablations
Mamba 1D conv	BF16	All layers
Router gates	FP32	Routing precision must not degrade
Embeddings & lm_head	BF16	Not quantized
KV cache	FP8	All 6 attention layers

Calibration

Dataset: 4,000 samples from nvidia/Nemotron-Cascade-2-SFT-Data
Domain mix: math (1000), swe (900), terminal_agent (500), science (500), chat (400), conversational_agent (300), instruction_following (300), safety (100)
Sequence length: Up to 12,288 tokens (no padding, natural length per sample)

Usage

SGLang

bash

python -m sglang.launch_server \
    --model chankhavu/Nemotron-Cascade-2-30B-A3B-FP8 \
    --trust-remote-code \
    --tool-call-parser qwen3_coder \
    --reasoning-parser nano_v3

vLLM