NVIDIA-Nemotron-3-Nano-30B-A3B — oQe Series

🚀 The oQe (Enhanced) Build Path

1. Expert Sensitivity Analysis: We identify which of the "Active-3" experts are most critical for specialized tasks (Code, Math, Reasoning) and protect their weights.
2. MoE Gating Protection: The router and attention heads are locked to a minimum of 6-bit or 8-bit precision to ensure token routing remains accurate.
3. Hessian-Based Tuning: Post-quantization adjustment is applied to the expert blocks to recover the "drift" caused by compressing the massive parameter space.

📋 oQ Build Performance Matrix

🛠 Technical Build Audit

• Calibration: Uses a 128-sample dataset ($128 \times 256$ tokens) covering the diverse language set (EN, ES, FR, DE, JA, IT).
• Sensitivity Proxy: Nano-30B-A3B-oQ8.
• MoE Strategy: Aggressive anchoring. We used 48 mandatory 8-bit anchors for the routing logic and early attention blocks to maintain the 30B parameter logic flow.

Model Highlights

• Active-3 MoE: Only 3 billion parameters are active per token, providing the inference speed of a small model with the broad knowledge base of a 30B model.
• Multilingual: Maintains strong performance across the 6 primary languages supported by the base model.
• Architecture: Hybrid MoE structure optimized for efficiency on Apple Silicon.