Views
No views yet
nvidia/nemotron-3.5-asr-streaming-0.6b,
so the streaming encoder (≈95 % of the compute) runs on the Apple Neural Engine while the prompt MLP
and RNN-T decoder stay in MLX on the GPU.mlx-community/nemotron-3.5-asr-streaming-0.6b
in mlx-audio-swift. For the offline encoder see
beshkenadze/nemotron-3.5-asr-streaming-0.6b-coreml-ane.1# auto-downloads this encoder and streams on the ANE:
2mlx-audio-swift-stt --model mlx-community/nemotron-3.5-asr-streaming-0.6b \
3 --audio in.wav --output-path out --stream --anetry await nemotron.enableCoreMLStreamingEncoder(repo: NemotronASRModel.defaultANEStreamingEncoderRepo).cache_aware_stream_step is converted functionally — the three streaming caches are explicit
inputs and outputs (no MLState), so Swift threads them across chunks (feed in → read new_* → feed
back). A fixed-shape ANE model can't honor a per-chunk true length, so the Swift wrapper feeds a uniform
window every chunk:processed_signal [1, 128, 121] = [9 prev-mel ++ 112 new-mel], stride 112 ·
cache_last_channel [24,1,56,1024] · cache_last_time [24,1,1024,8] · cache_last_channel_len [1] (i32).encoded [1,1024,14] fp16 + the three new_* caches. att_context [56,13].palettize_weights, uniform): 561 MB (≈2× smaller than fp16) and
~28 % faster on the ANE (~19 ms/chunk vs ~27 ms fp16) — and the streamed transcript is word-for-word
identical to fp16/MLX. (6-bit was faster still but degraded the transcript, so 8-bit is shipped.)MLComputeUnits.cpuAndNeuralEngine) — the few int32 mask /
cache-length ops drop to CPU as one negligible island. Use CPU+ANE, not .all: with .all,
CoreML places the whole graph on the GPU (≈2 % ANE) — no power win..mlmodelc (cold compile once ~33 s, then hot start ~2.3 s — within 0.4 s of pure MLX).MLModel + MLComputeUnits only — no private APIs.tools/coreml-ane/convert_encoder_coreml_stream.py --model nvidia/nemotron-3.5-asr-streaming-0.6b --att-context 56 13
(needs NeMo ≥ 2.8 for EncDecRNNTBPEModelWithPrompt; patches coremltools' aten::Int to convert on torch 2.10).