Views
No views yet
nvidia/diar_streaming_sortformer_4spk-v2
(cc-by-4.0, 117M) converted to Apple Core AI — streaming speaker diarization ("who spoke
when", up to 4 speakers) running fully on-device via the
zoo. Only the neural core is a graph; the NeMo
128-mel frontend, the streaming chunk loop, and the AOSC speaker-cache compression run in the Swift
host — a 1:1 port of NeMo sortformer_modules.py (inference path).diar_sortformer_4spk-v1 is CC-BY-NC.sortformer_float16.aimodel — the static forward_for_export core, fp16 (~237 MB, macOS GPU).sortformer_float16.h18p.aimodelc — the same graph AOT-compiled for iPhone (h18p, ~450 MB).sortformer_mel_filters_128x257.f32 — librosa-slaney mel filterbank (host log-mel frontend).metadata.json — streaming params + the fixed-buffer graph contract.inputs: chunk_mel [1,1520,128] host zero-pads each mel chunk
spkcache [1,188,512] host-maintained speaker cache
valid [1,378] 1 = real frame / 0 = pad (spkcache block [0:len], chunk block [188:188+pe_len])
outputs: preds [1,378,4] sigmoid speaker activity
chunk_pe [1,190,512] pre-encode embeddings (host appends them to the speaker cache)streaming_update + compress_spkcache (AOSC) → threshold 0.5/frame/speaker (frame = 80 ms) → turns.forward_streaming at 100.00 % speaker-activity agreement (@0.5) on a 21.5 s
and a 64.5 s clip (the latter exercises the AOSC cache compression ~4×), in Python, in Swift on
Mac GPU, and on iPhone 17 Pro (A19 Pro, AOT h18p) — all driving this exported fp16 graph.Speaker 1 [0.3–4.1s]: …. Speaker diarization already ships
on-device elsewhere (e.g. CoreML/ANE); this is speed parity, offered as a diarized transcript wired to
the zoo's own ASR. Conversion + Swift host loop: see
conversion/sortformer_diar.diar_streaming_sortformer_4spk-v2 (CC-BY-4.0); this Core AI conversion is
released under the same license.