Speaker Diarization Models · Who Spoke When (Fully Local)
Give it a multi-speaker conversation recording and this model marks who said each segment (Speaker 1, Speaker 2, …) — entirely on your own computer. No cloud, free, private.
The models were built by two open-source teams:
pyannote (speech segmentation, MIT) and
WeSpeaker (speaker embeddings, Apache-2.0); the ONNX deployment packaging follows
avencera/speakrs (Apache-2.0).
This repo is not a new model — we converted these open models into deployment builds that
run offline on your computer and verified their quality. It is an
enhancement model and does not transcribe: transcription is still done by
Qwen3-ASR, and this model adds the "who is speaking" information on top.
1. The problem it solves
Speech-to-text tells you "what was said", but in a multi-speaker conversation it often cannot tell you who said this line. Many scenarios depend precisely on that distinction:
- Meeting minutes: organize notes by speaker so each party's position is visible at a glance;
- Interviews and podcasts: separate host from guest, making transcripts and quotations easier to prepare;
- Customer service and QA: distinguish agent from customer and analyze each separately.
Doing speaker diarization used to mean uploading the recording to a cloud service — with both privacy concerns and possible costs. This model keeps that step entirely local.
2. How it works
It chains three steps into a "who spoke when" pipeline:
- Segmentation (pyannote): first find which stretches of audio contain speech, and where the speaker changes;
- Speaker embeddings (WeSpeaker): extract a "voiceprint" vector representing the voice characteristics of each short segment;
- Clustering: group segments with similar voiceprints as the same person, and finally attach a speaker label to each segment.
What we did: converted the upstream open weights into locally runnable deployment builds, verified that quality did not degrade in conversion, and made them run offline across macOS / Windows / Linux; we also selected the right acceleration path for Apple silicon (CoreML), NVIDIA GPUs (CUDA) and plain CPU, so machines without a discrete GPU can use it too.
3. How well it performs
For the complete diarization-quality benchmarks (DER, lower is better), refer to upstream public evaluations — per
speakrs benchmarks, this pipeline (CoreML) achieves about
7.1% DER on the VoxConverse
Dev subset, essentially on par with pyannote community-1 (about 7.2%). Here we list only
our own verification:
| What we verified | Result |
|---|
| Correctness on a Chinese two-person conversation (5-minute real recording) | Correctly separated 2 speakers, with sensible speaker-change points |
| Speed · Apple silicon (CoreML) | ~380× real time (5 minutes of audio in about 0.8 s) |
| Speed · plain CPU | ~1.9× real time (5 minutes of audio in about 2.6 minutes), usable on machines with no GPU |
| Memory use (plain CPU) | About 1.2 GB |
Note: "correctness", "speed" and "memory" above are our own measurements on an Apple M3 Max; "7.1% DER (VoxConverse Dev)" is a public figure from upstream speakrs (a third-party CoreML port's evaluation — neither pyannote official nor measured by us).
4. Limitations and what's next
- Requires a transcription model: this model only answers "who is speaking" and does not transcribe; use it together with Qwen3-ASR.
- Plain-CPU speed: extremely fast with Apple / NVIDIA acceleration; about 1.9× real time on plain CPU — usable, but clearly slower than the accelerated tiers, so process long recordings on an accelerated machine where possible.
- Overlapping speech: where two people talk at once, speaker attribution may be imprecise (a shared difficulty of speaker diarization).
- Next: keep improving plain-CPU speed and handling of overlapping segments.
5. How to download and use it
This model is packaged for
42model, which is the recommended way to get it:
Desktop app
- Open Model Library → Transcription and download Qwen3-ASR Enhanced (speaker diarization is built in);
- When transcribing, turn on Speaker diarization in the parameter settings, and results will be segmented by speaker.
Files and license
| File | Role | Source · License |
|---|
segmentation-3.0.onnx | Speech segmentation / speaker-change detection | pyannote/segmentation-3.0 · MIT |
wespeaker-voxceleb-resnet34.onnx | Speaker embedding extraction | WeSpeaker voxceleb ResNet34 · Apache-2.0 |
plda_lda.npy, plda_tr.npy, plda_mu.npy, plda_psi.npy, plda_mean1.npy, plda_mean2.npy | Voiceprint clustering parameters (PLDA / VBx, 6 files) | speakrs pipeline · Apache-2.0 |
manifest.json | sha256 / size listing for each file (generated at release, for integrity self-verification) | Generated by this repo |
Per-file sha256 values are in the bundled manifest.json and on the Files page, and can be verified independently. CoreML acceleration on Apple silicon is compiled automatically on first local run — no extra files needed.
License: this repo contains local deployment builds of open models from several sources — speech segmentation is
pyannote/segmentation-3.0 (© pyannote.audio,
MIT; the upstream HF page requires accepting terms for access, but the license is MIT and redistribution is unrestricted — the ONNX here comes from the same weights under the same license); speaker embeddings are
WeSpeaker voxceleb ResNet34 (© the WeNet community,
Apache-2.0); the ONNX packaging and pipeline follow
avencera/speakrs (Apache-2.0). Each file is governed by its original license; by using them you agree to the respective upstream license terms.
Citation
For the models themselves, please cite upstream pyannote and WeSpeaker:
1@inproceedings{Plaquet23,
2 author = {Alexis Plaquet and Herv{\'e} Bredin},
3 title = {{Powerset multi-class cross entropy loss for neural speaker diarization}},
4 booktitle = {Proc. INTERSPEECH 2023},
5 year = {2023},
6}
7@inproceedings{Bredin23,
8 author = {Herv{\'e} Bredin},
9 title = {{pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe}},
10 booktitle = {Proc. INTERSPEECH 2023},
11 year = {2023},
12}
13@inproceedings{wang2023wespeaker,
14 author = {Wang, Hongji and Liang, Chengdong and Wang, Shuai and Chen, Zhengyang and Zhang, Binbin and Xiang, Xu and Deng, Yanlei and Qian, Yanmin},
15 title = {{Wespeaker: A research and production oriented speaker embedding learning toolkit}},
16 booktitle = {ICASSP 2023},
17 year = {2023},
18}
If this repo's local deployment packaging was useful to you, you may additionally cite:
1@misc{yang2026diarizationlocal,
2 title = {Speaker Diarization Models: A Local Build of pyannote + WeSpeaker for On-Device Diarization},
3 author = {Yang, Zhiping},
4 year = {2026},
5 howpublished = {\url{https://huggingface.co/42ailab/Speaker-Diarization-Models}},
6 organization = {42ailab},
7 note = {Local deployment packaging (ONNX + cross-platform acceleration); the models themselves are pyannote/segmentation-3.0 (MIT) and WeSpeaker voxceleb ResNet34 (Apache-2.0). Contact: contact@42ailab.com}
8}
About us
42ailab — an AI research lab exploring the boundaries of intelligence. Grounded in cognitive science, we work toward a deep integration of AI and human intelligence — to truly understand and augment intelligence, carbon-based and silicon-based alike.
42model — a high-performance local inference engine from 42ailab that runs translation, transcription, recognition, chat and coding on your own machine, free and private; with optional cloud compute for fine-tuning your own models and bringing them back to run locally.
