Three lightweight CNN-based binary classifiers that detect waveform-domain artifacts in TTS-generated audio. These models identify vocoder artifacts (metallic resonance, buzzing) and comb-filtering artifacts that emerge from neural vocoders such as BigVGAN/HiFi-GAN.
All models operate on 16 kHz mono waveforms and output an artifact score in [0, 1]: 0 = clean, 1 = artifact detected.
Models
Model
Architecture
Parameters
Val Accuracy
Val F1
File
STFT Classifier
Multi-resolution STFT + 2D CNN
1,688,833
99.88%
99.87%
best_stft_classifier.pt
Mel CNN
Mel spectrogram + 2D CNN
1,012,417
99.75%
99.75%
best_mel_classifier.pt
Waveform 1D
Raw waveform 1D CNN
1,898,753
97.44%
97.39%
best_waveform_1d.pt
Per-Source Validation Accuracy
Each model was validated on three source types:
Model
Predicted (vocoder artifacts)
Real (clean audio)
Comb (synthetic comb filter)
STFT Classifier
100%
100%
99.0%
Mel CNN
100%
100%
98.0%
Waveform 1D
99.8%
99.4%
82.5%
Quick Start
Installation
pip install torch torchaudio soundfile numpy
Python API
python
1from artifact_detector import load_all_models, score_file, score_file_all
23# Load all 3 models4models = load_all_models(".", device="cuda")56# Score a single file with all models7scores = score_file_all(models,"my_tts_output.wav")8for name, score in scores.items():9print(f" {name}: {score:.4f}")10# → stft_classifier: 0.982311# → waveform_1d: 0.890112# → mel_classifier: 0.00121314# Ensemble average15import numpy as np
16ensemble = np.mean(list(scores.values()))17print(f" ensemble: {ensemble:.4f}")1819# Single model20from artifact_detector import load_model, score_file
21model = load_model("best_stft_classifier.pt", device="cuda")22score = score_file(model,"my_tts_output.wav")
Command Line
bash
1# Score a file with all models2python artifact_detector.py --model-dir . --input audio.wav
34# Score a directory of WAV files5python artifact_detector.py --model-dir . --input /path/to/wavs/ --ext wav
67# Score with a specific model only8python artifact_detector.py --checkpoint best_stft_classifier.pt --input audio.wav
910# Custom threshold (default 0.5)11python artifact_detector.py --model-dir . --input audio.wav --threshold 0.3
Scoring Raw Tensors
python
1import torch
2from artifact_detector import load_model, score_waveform
34model = load_model("best_stft_classifier.pt")56# waveform should be a 1-D float32 tensor at 16 kHz7waveform = torch.randn(160000)# 10 seconds8score = score_waveform(model, waveform)
Using as a Differentiable Loss
All models are fully differentiable and can be used as auxiliary training losses:
The best-performing model. Computes STFT at 4 resolutions (256/512/1024/2048 FFT sizes), processes each through a 5-layer 2D CNN on log-magnitude spectrograms, concatenates the 128-dim feature vectors from each resolution (512 total), and classifies through an MLP head.
Input: [B, 1, T] mono waveform at 16 kHz
STFT computed internally (fully differentiable)
4 resolution blocks with adaptive average pooling
MLP head: 512 → 256 → 1
2. Waveform 1D CNN
Direct 1D convolution on raw waveform samples. 6 conv blocks with channels [1→64→128→256→256→512→512], kernel sizes [15, 11, 7, 5, 3, 3], BatchNorm, GELU activation, and MaxPool downsampling. Global average pooling feeds into a 2-layer MLP head.
Input: [B, 1, T] mono waveform at 16 kHz
No spectral transform needed
Largest model (1.9M params) but lowest accuracy
3. Mel CNN Classifier
Computes an 80-band mel spectrogram from the raw waveform (differentiable), then runs through a 5-layer 2D CNN with BatchNorm, GELU, and MaxPool2d. Adaptive average pooling feeds into a 2-layer MLP head.
6,000 TTS-predicted audio decoded through a BigVGAN-based vocoder at 3 noise levels (σ = 0.15, 0.275, 0.4)
2,000 synthetic comb-filtered versions of the clean audio (applied on-the-fly with random delay 0–6ms and wet mix 0.3–0.95)
Total: 16,000 samples per epoch. 90/10 train/val split, stratified by source type.
Training Configuration
Optimizer: AdamW (lr=3e-4, weight_decay=1e-4)
Scheduler: CosineAnnealingLR
Loss: BCEWithLogitsLoss
Batch size: 32
Max audio length: 10 seconds (160,000 samples)
Gradient clipping: max_norm=1.0
30 epochs, best checkpoint selected by validation accuracy
Checkpoint Format
Each .pt file contains:
python
1{2"model_state_dict":...,3"architecture":"stft_classifier",# or "waveform_1d" or "mel_classifier"4"epoch":25,5"val_acc":0.9988,6"val_f1":0.9987,7"n_params":1688833,8"input_sr":16000,9}
Intended Use
Quality filtering of TTS training data
Automated QA for text-to-speech pipelines
Differentiable auxiliary loss for vocoder fine-tuning
Detection of comb-filter artifacts in audio processing chains
Limitations
Trained on a specific vocoder architecture (BigVGAN-based). May not generalize to all TTS systems without fine-tuning.
Models disagree on some samples — the STFT model is most reliable overall, while Waveform1D tends to have higher false positive rates.
Not trained to detect other audio quality issues (clipping, noise, bandwidth limitation).
10-second maximum context window; longer files are truncated.
