You can deploy this model and start earning money today!

aufklarer/
Nemotron-3.5-ASR-Streaming-0.6B-MLX-8bit

Name: Nemotron-3.5-ASR-Streaming-0.6B-MLX-8bit
Availability: InStock

mlx

safetensors

nemotron_streaming_rnnt_mlx

automatic-speech-recognition

Views

No views yet

Nemotron-3.5 ASR Streaming 0.6B — MLX INT8

Cache-aware multilingual Conformer + RNN-T speech recognition from NVIDIA, converted to native MLX for streaming inference on Apple silicon. This bundle uses affine group-64 8-bit linear weights, BF16 recurrent state, and unquantized convolution, embedding, LSTM, and language-prompt layers.

Model


Parameters	0.6B
Architecture	Cache-aware FastConformer + prompt-conditioned RNN-T
Sample rate	16 kHz mono
Streaming chunk	320 ms
Encoder frame rate	80 ms
Attention left context	56 frames (4.48 s)
Vocabulary	13,087 SentencePiece tokens + blank
Quantization	MLX affine group-64 8-bit linear weights
On-disk size	732.6 MB

Files

File	Size	Description
`model.safetensors`	732.1 MB	MLX model weights
`config.json`	1.7 KB	Strict runtime and streaming geometry
`vocab.json`	100.9 KB	Ordered SentencePiece vocabulary
`languages.json`	2.0 KB	Wrapped language-to-prompt-slot mapping
`lang2slot.json`	1.7 KB	Flat compatibility language mapping
`tokenizer.model`	397.0 KB	Original SentencePiece model
`speech_models_export.json`	1.3 KB	Pinned source and export artifact hashes
`checksums.json`	generated	SHA-256 for every published file

Streaming accuracy

FLEURS test, 50 samples per language, cache-aware 320 ms streaming. English uses Whisper's EnglishTextNormalizer; German, French, and Arabic use BasicTextNormalizer; Hindi and Japanese use BasicTextNormalizer(split_letters=True).

Language	WER %	CER %	RTF
en-US	8.98	3.96	0.0444
de-DE	10.59	5.73	0.0439
fr-FR	11.83	4.66	0.0451
ar	13.37	3.77	0.0481
hi-IN	4.28	3.50	0.0547
ja-JP	17.01	11.42	0.0559

Mean WER across these six slices is 11.01% and mean CER is 5.50%.

Streaming performance

M5 Pro with 48 GB unified memory, one fresh Python process, 63.7 seconds of concatenated FLEURS English audio, 199 × 320 ms chunks:

metric	value
Real-time factor	0.0485
p50 chunk latency	14.3 ms
p95 chunk latency	16.2 ms
p99 chunk latency	18.9 ms
RSS after lazy weight load	196 MB
Peak RSS during streaming	992 MB

The post-load value reflects MLX's lazy memory mapping; peak RSS is the useful resident-memory figure after inference touches the weights.

Swift usage

The native runtime is in NemotronStreamingASR.

swift

import NemotronStreamingASR

let model = try await NemotronStreamingASRMLXModel.fromPretrained(
    variant: .int8
)
let session = try model.createSession(language: "en-US")

for pcm320ms in audioChunks {
    for partial in try session.pushAudio(pcm320ms) {
        print(partial.text)
    }
}

for final in try session.finalize() {
    print(final.text)
}

Sessions retain the mel, attention, convolution, and RNN-T predictor caches. Model calls are serialized so multiple source-local sessions can safely share one loaded weight set.

Python reference runtime

The reproducible exporter, cache-aware runtime, tests, and benchmarks live in speech-models/models/nemotron-asr-streaming-multilingual/export.

bash

hf download aufklarer/Nemotron-3.5-ASR-Streaming-0.6B-MLX-8bit --local-dir ./nemotron-mlx

python streaming.py \
  --bundle ./nemotron-mlx \
  --wav recording.wav \
  --lang en-US \
  --chunk-ms 320 \
  --print-partials

--print-partials prints cumulative text whenever a chunk adds visible tokens. Both reference runtimes remove angle-bracket language markers such as <en-US> from user-facing text.

Choosing a precision

INT5 minimizes resident memory; INT8 preserves more quantization headroom. Benchmark both on domain audio before deployment.

Limitations

This is ASR only. It does not perform speaker diarization or speaker identification.
Language conditioning is explicit. Pass a supported BCP-47 language tag or use the bundle's auto slot.
Partial text is provisional and may end mid-word. Applications should replace the previous partial until the caller finalizes the utterance.
The native MLX path does not currently implement the Core ML runtime's optional word-boosting layer.
The runtime is intended for Apple silicon and requires the MLX Metal shader library to be packaged with the executable.

Provenance


Upstream repository	`nvidia/nemotron-3.5-asr-streaming-0.6b`
Upstream revision	`f3d333391852ba876df169dcc9ba902d25b6ab0b`
Upstream `.nemo` SHA-256	`210214ed94039bf6bfbb9a047c7fa289628db75b103e2bf6381fa78285436a74`
Export manifest	`speech_models_export.json`
Artifact checksums	`checksums.json`

License

Distributed under OpenMDW 1.1, matching the upstream NVIDIA model. Review the upstream model card and license before deployment.

aufklarer/Nemotron-3.5-ASR-Streaming-0.6B-MLX-8bit