An MLX quantization of nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, produced with oMLX's quantizer (hence the oQ8e suffix).

Uniform 8-bit, group size 64, affine mode. There are no per-module precision overrides — every quantized module, including all 128 routed experts, the shared experts, backbone.embeddings, and lm_head, is at 8 bits.

Quantization was imatrix-guided. The calibration pass used oMLX's oqe_code_multilingual set (128 samples at sequence length 512) and reached full expert coverage — all 5888 routed experts were activated during calibration, with no zero-count experts.

Resulting size on disk: ~31 GiB.

A mixed-precision 4-bit conversion of the same upstream model, from the same calibration run, is available at NVIDIA-Nemotron-3.5-Lightning-30B-A3B-mlx-oQ4e.

The upstream model is a hybrid: Mamba-2 state-space layers, MoE layers, and attention layers interleaved — 52 layers total (23 mamba, 23 MoE, 6 attention), with 128 routed experts, 6 active per token, plus 1 shared expert. model_type is nemotron_h; architectures is ["NemotronHForCausalLM"].

Upstream describes MTP (multi-token prediction) layers, and its deployment guidance targets vLLM on CUDA. This conversion contains no MTP weights — config.json reports num_nextn_predict_layers: 0 and the safetensors index holds no MTP tensors. MTP and speculative decoding have not been exercised here at all.

This quant has been tested in oMLX only. No claims are made about behavior under any other runtime, and no benchmarks or quality evaluations have been run against it.

All model weights and design credit belong to NVIDIA. This repository redistributes a quantized conversion of their release. See the upstream model card for the model's capabilities, intended use, training details, and license terms: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16