Views
No views yet
| File | Size | Purpose |
|---|---|---|
embedding_encoder.onnx | 20 MB | ResNet encoder, output frame features (B, 2560, F) |
resnet_seg_1_weight.npy | 5 MB | Final Gemm projection weight (256, 5120) |
resnet_seg_1_bias.npy | 1 KB | Final Gemm projection bias (256,) |
split_pyannote_embedding.py | — | Reproducibility script |
embedding_model.onnx graph is fbank → ResNet → StatsPool → Gemm → 256d. The internal StatsPool op pools all frames of the input — there's no way to mask which frames belong to which speaker.| Approach | Speed | Correctness |
|---|---|---|
| Run full model once per (chunk, speaker), filtering frames first | ~720 chunks × 3 speakers = 2160 ORT calls for a 2-hour file | ⚠️ Wrong: subset-filtering is not the same as weighted pooling — variance is computed on a different denominator |
| This split: batch-encode 64 chunks at a time, then NumPy-pool with mask | ~12 ORT calls for 2 hours | ✅ Matches pyannote.audio.models.blocks.pooling.StatsPool._pool() |
run_with_iobinding and enable_cpu_mem_arena=False, this gives ~30× speedup on CPU for the embedding extraction stage of long-form diarization.1import numpy as np
2import onnxruntime as ort
3from huggingface_hub import snapshot_download
4
5local = snapshot_download("welcomyou/pyannote-community-1-onnx-split")
6
7opts = ort.SessionOptions()
8opts.enable_cpu_mem_arena = False # avoid 1.8 GB arena that never shrinks
9encoder = ort.InferenceSession(f"{local}/embedding_encoder.onnx", opts,
10 providers=["CPUExecutionProvider"])
11W = np.load(f"{local}/resnet_seg_1_weight.npy") # (256, 5120)
12b = np.load(f"{local}/resnet_seg_1_bias.npy") # (256,)
13
14
15def masked_stats_pool(frame_feat, mask):
16 """frame_feat: (D=2560, F), mask: (F,) float [0..1] — per-frame speaker weight.
17 Returns (5120,) = concat(weighted_mean, weighted_std).
18 Matches pyannote.audio StatsPool._pool()."""
19 w = mask[np.newaxis, :]
20 v1 = w.sum() + 1e-8
21 mean = (frame_feat * w).sum(axis=1) / v1
22 var = ((frame_feat - mean[:, None]) ** 2 * w).sum(axis=1) / (v1 - (w * w).sum() / v1 + 1e-8)
23 return np.concatenate([mean, np.sqrt(var)])
24
25
26# Batched encoder pass: (64 chunks, 998 frames, 80 dim fbank)
27fbank_batch = ... # shape (64, 998, 80) float32
28frame_feats = encoder.run(None, {"fbank_features": fbank_batch})[0] # (64, 2560, F)
29
30# For each (chunk, speaker) compute embedding:
31for c in range(64):
32 for s in range(num_speakers):
33 mask = per_speaker_activity_masks[c, s] # (F,) float32
34 stats = masked_stats_pool(frame_feats[c], mask)
35 emb = stats @ W.T + b # (256,)core/speaker_diarization_pure_ort.py (lines 707–810).1# 1. Download the upstream ONNX export from altunenes
2huggingface-cli download altunenes/speaker-diarization-community-1-onnx \
3 --include embedding_model.onnx --local-dir pyannote-onnx/
4
5# 2. Run the split script (≈30 s)
6python split_pyannote_embedding.py \
7 --input pyannote-onnx/embedding_model.onnx \
8 --output_dir pyannote-onnx/onnx.utils.extract_model to carve out the encoder subgraph (output tensor /resnet/pool/Reshape_output_0, just before stats pooling) and onnx.numpy_helper to dump the final Gemm initializers resnet.seg_1.weight and resnet.seg_1.bias as .npy.pyannote/speaker-diarization-community-1). Attribution required; commercial use allowed.pyannote/speaker-diarization-community-1 repository is gated and requires accepting a contact-information form. CC-BY-4.0 itself does not impose that restriction on derivative works, but please consider visiting the original repo to support pyannote.pyannote.audio (~2.1 GB → ~60 MB of dependencies).