DiariZen Speaker Segmentation - AX650 Deployment

Model

• Source: BUT-FIT/diarizen-wavlm-large-s80-md
• Architecture: WavLM-Large (pruned) + Conformer
• Task: Frame-level speaker activity segmentation (11 classes, 4s @ 16kHz)

Pipeline

Audio (16kHz mono, any length)
  → CPU: resample → 4s sliding window → LayerNorm
  → AX650 NPU: CNN feature extractor (7 conv, U16, 17.7ms)
  → CPU: WavLM Transformer (24L) + Conformer (4L) + Classifier (251ms)
  → Log-probabilities (1, 199, 11) per window

Performance

Accuracy

Directory

models/          cnn_features.axmodel + backend.onnx
python/          Python SDK (diarizen_sdk)
cpp/             C++ SDK (diarizen_segmenter)
model_convert/   ONNX export + Pulsar2 compile config
reports/         SDK and simulation reports

Quick Start

Known Limitations

• 4s fixed window; longer audio requires sliding window stitching.
• CNN requires U16 quantization (U8 insufficient accuracy).
• Transformer + Conformer run on CPU (NPU backend limitation with WavLM attention ops).
• pos_conv_embed Conv (kernel=128, NLC layout) incompatible with NPU; included in CPU backend.