You can deploy this model and start earning money today!

AdityaAA2004/
dagger-phase1-proposed-librimix-3spk

pytorch

audio

speech-separation

target-speech-extraction

Views

No views yet

dagger — Phase 1 "proposed" extractor (LibriMix, 3-speaker)

Trained checkpoint for the dagger project's Phase 1 target-speech-extraction system: a TF-GridNet + cross-attention extractor G(x_O, ē_i) conditioned on a speaker embedding, which pulls one speaker out of an overlapping-speech mixture using the untouched original mixture at every step (never a subtracted residual — see the project's central claim in CLAUDE.md §1).

Repo / code: https://github.com/RohanBanerjee88/dagger
Architecture: TF-GridNet + cross-attention fusion (dagger/extract/), original implementation informed by the USEF-TSE paper (arXiv:2409.02615); not vendored from USEF-TSE or WeSep.
Training config: configs/phase1_librimix_3spk_train.yaml (--system proposed) — 2000 Libri3Mix train-360 scenes, 30 epochs, batch size 4, lr 3e-4, single T4 GPU.
Eval config: configs/phase1_librimix_3spk_eval.yaml (150 Libri3Mix test scenes, oracle diarization).
Result: mean overlap SI-SDR 4.40 dB, vs. a blind-separation baseline at 2.05 dB (+2.35 dB), 450 speaker-rows. See the repo's Results section and results/phase1_librimix_3spk.csv for the full per-scene table and caveats (per-row win rate is 50% — the mean margin comes from win/loss magnitude asymmetry, not a majority of rows winning; see the repo README for the honest breakdown).
Sample rate: 8 kHz (dev-scale LibriMix setting used throughout Phase 1/2).

Files

proposed_librimix_3spk.pt — PyTorch state dict for the extractor G. Load with torch.load(..., map_location=...) and the extractor class in dagger/extract/tfgridnet_crossattn.py; see the "Usage" section below.

License

This checkpoint (the extractor G's trained weights) is released under Apache-2.0, matching the dagger repository's license.

Important dependency note: at inference time, dagger also loads NVIDIA's pretrained TitaNet-Large speaker-verification checkpoint (nvidia/speakerverification_en_titanet_large) as the frozen speaker encoder φ, via the NVIDIA NeMo toolkit. TitaNet-Large is licensed CC-BY-4.0 by NVIDIA and is a separate model — it is NOT bundled in this HF repo, and this repo's Apache-2.0 license applies only to the extractor weights uploaded here. NeMo itself (the loading toolkit) is Apache-2.0. See the dagger repo's NOTICE file for the canonical attribution text.

Usage

python

from huggingface_hub import hf_hub_download
import torch

ckpt_path = hf_hub_download(
    repo_id="AdityaAA2004/dagger-phase1-proposed-librimix-3spk",
    filename="proposed_librimix_3spk.pt",
)
state_dict = torch.load(ckpt_path, map_location="cpu")
# then load into dagger.extract.tfgridnet_crossattn.TFGridNetCrossAttnExtractor
# — see https://github.com/RohanBanerjee88/dagger for the full pipeline
# (diarization -> enrollment -> extraction -> reconstruction).