Native CoreML conversion of nvidia/nemotron-speech-streaming-en-0.6b, a 600M-parameter streaming ASR model using FastConformer encoder + RNNT decoder. Optimized for Apple Neural Engine (ANE) on Apple Silicon.
Converted directly from the original NeMo checkpoint via coremltools, with 3-level numerical validation against the PyTorch reference.
Mel spectrogram preprocessing runs on the host (not in CoreML).
Important: ANE stride padding
CoreML output MLMultiArrays may have non-contiguous strides due to ANE alignment padding. For example, the encoder output [1, 1024, 7] may have physical strides [32768, 32, 1] instead of C-contiguous [7168, 7, 1]. Callers must use stride-aware copy, not flat memcpy.
Runtime Configuration
All parameters needed to run the model are documented in config.json, including I/O specs, cache shapes, the streaming protocol, and ANE profiling results.
Audio Preprocessing
Parameter
Value
Sample rate
16000 Hz
Sample format
S16_LE (16-bit signed little-endian)
Pre-emphasis
0.97
FFT size
512
Hop length
160 samples (10ms)
Window length
400 samples (25ms)
Window type
Hann
Mel bands
128
Mel norm
Slaney
Mel layout
Band-major[n_mels, n_frames] (not frame-major)
Encoder Streaming
Parameter
Value
Chunk size
56 mel frames (560ms audio)
Pre-encode cache
9 mel frames (prepended from previous chunk)
Total input frames
65 per chunk (56 + 9)
Layers
24
Dimension
1024
cache_last_channel shape
[1, 24, 70, 1024] FP32 in, FP16 out (init zeros)
cache_last_time shape
[1, 24, 1024, 8] FP32 in, FP16 out (init zeros)
cache_last_channel_len
[1] int32 (init zero)
Feed cache outputs back as next chunk's cache inputs. Convert FP16 outputs to FP32 before feeding back (model expects FP32 inputs).
RNNT Decoder
Parameter
Value
Blank token ID
1024
Vocab size
1025 (1024 tokens + blank)
Max symbols per frame
10
Prediction layers
2 (LSTM)
Prediction hidden
640
input_states_1/2 shape
[2, 1, 640] FP32 in, FP16 out (init zeros)
For each encoder output frame: feed single frame [1, 1024, 1] to decoder, argmax logits over 1025 vocab, if not blank emit token and loop (up to 10), if blank move to next frame. Feed decoder states back for next symbol/frame.
