Nemotron-Labs-Audex-30B-A3B NVFP4
An independently produced hybrid NVFP4 quantization of
nvidia/Nemotron-Labs-Audex-30B-A3B,
created with
NVIDIA Model Optimizer.
The routed MoE experts are compressed to NVFP4 while the audio system and
precision-sensitive components remain in BF16. This is an experimental build,
not an official NVIDIA release.
Repository layout
This repository mirrors the terminology used by NVIDIA's original Audex
distribution:
checkpoint_folder_full - audio understanding, transcription, and AudioQA
(the "ears").checkpoint_folder_audiogen - text-to-speech and text-to-audio generation
(the "voice").audex_causal_speech_decoder - converts generated speech-codec tokens into
the final waveform.inference_scripts_vllm - task-specific vLLM inference scripts.
The original root-level checkpoint_folder_full files remain temporarily for
backward compatibility. New integrations should use the canonical folder paths
above.
Verified locally
- GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB)
- Checkpoint size: 21.47 GiB on disk
- Model loading VRAM: 22.18 GiB
- Load time: 24 seconds, including graph compilation
- Warm short-response latency: 53.9 ms average (40.5-92.9 ms)
- Deterministic stability: 15/15 exact responses
- Exact speech transcription passed before and after a full restart
- Audio-understanding request passed
- AudioGen NVFP4 loaded with the native Blackwell NVFP4 MoE backend
- Native TTS validation produced a 5.34-second, 16 kHz WAV
The AudioGen checkpoint requires enforce_eager=True with the tested vLLM
0.24.0 WSL stack. Its measured model-loading footprint was 20.96 GiB VRAM.
See
audex-nvfp4-build-report.pdf for the
test commands, measurements, hashes, and qualification notes.
Intended use
Research and internal experimentation with audio understanding, speech
recognition, and multimodal language-model serving on NVIDIA Blackwell GPUs.
License
The upstream model is distributed under NVIDIA's OneWay Noncommercial License.
Review the upstream license before downloading, redistributing, or deploying
this derivative checkpoint. No additional commercial rights are granted here.
