You can deploy this model and start earning money today!

panlr/
BigVGAN_24khz_teochew

Name: BigVGAN_24khz_teochew
Availability: InStock

nvidia/bigvgan_v2_24khz_100band_256x

finetune

cc-by-4.0

Views

No views yet

license: Apache License 2.0

BigVGAN (24khz) Vocoder for Teochew TTS

模型简介 (Model Description)

本模型是基于 BigVGAN 架构训练的潮州话语音合成声码器，用于从梅尔频谱重建高质量音频波形。模型基于 NVIDIA BigVGAN 训练代码，从预训练权重进行微调 (fine-tuning)。

(This is a BigVGAN vocoder trained for Teochew speech synthesis, designed to reconstruct high-quality audio waveforms from mel-spectrograms. The model was fine-tuned from pre-trained weights using NVIDIA's BigVGAN training code.)

预训练权重 (Pre-trained Weights)

基础模型 (Base Model): nvidia/bigvgan_v2_24khz_100band_256x

训练数据 (Training Data)

数据集 (Dataset): teochew-extLa
筛选标准 (Filter Criteria): DNSMOS > 2.8
数据量 (Data Size): 约 283,000 条音频 (~283,000 audio clips)
总时长 (Total Duration): 555+ 小时 (hours)

训练配置 (Training Configuration)

硬件环境 (Hardware):

8 × V100 GPU
Batch Size: 8

训练迭代 (Training Iterations):

Iterations: 400,000（约 11 个 epochs / ~11 epochs, very slow）

训练时长 (Training Duration): 约 9 天 (~9 days)

超参数 (Hyperparameters):

Learning Rate: 0.0001
Adam β1: 0.8
Adam β2: 0.99
LR Decay: 0.9999996
Gradient Clip Norm: 500

模型架构 (Model Architecture)

生成器配置 (Generator Config):

python

{
    "upsample_rates": [4,4,2,2,2,2],
    "upsample_kernel_sizes": [8,8,4,4,4,4],
    "upsample_initial_channel": 1536,
    "resblock_kernel_sizes": [3,7,11],
    "resblock_dilation_sizes": [[1,3,5], [1,3,5], [1,3,5]],

    "use_tanh_at_final": false,
    "use_bias_at_final": false,

    "activation": "snakebeta",
    "snake_logscale": true,
}

判别器配置 (Discriminator Config):

python

{
    "use_cqtd_instead_of_mrd": true,
    "cqtd_filters": 128,
    "cqtd_max_filters": 1024,
    "cqtd_filters_scale": 1,
    "cqtd_dilations": [1, 2, 4],
    "cqtd_hop_lengths": [512, 256, 256],
    "cqtd_n_octaves": [9, 9, 9],
    "cqtd_bins_per_octaves": [24, 36, 48],

    "mpd_reshapes": [2, 3, 5, 7, 11],
    "use_spectral_norm": false,
    "discriminator_channel_mult": 1,
}

梅尔频谱配置 (Mel-Spectrogram Config):

python

{
    "sampling_rate": 24000,
    "n_fft": 1024,
    "num_mels": 100,
    "hop_size": 256,
    "win_size": 1024,
    "fmin": 0,
    "fmax": null,
    
    "segment_size": 65536,
    "use_multiscale_melloss": true,
    "lambda_melloss": 15
}

使用方法 (Usage)

1. 命令行界面 (Command Line Interface)

bash

cd bigvgan_standalone

# 单文件重构 (Single file reconstruction)
python bigvgan_api.py --config_dir exp/bigvgan_teochew_24khz --checkpoint g_05130000 \
    --input audio.wav --output out.wav

# 批量处理 (Batch processing)
python bigvgan_api.py --config_dir exp/bigvgan_teochew_24khz --checkpoint g_05130000 \
    --input a.wav b.wav c.wav --output_dir results/

# 从梅尔频谱重构 (Reconstruct from mel-spectrogram)
python bigvgan_api.py --config_dir exp/bigvgan_teochew_24khz --checkpoint g_05130000 \
    --mel mel.npy --output out.wav

# 从文件列表处理 (Process from file list)
python bigvgan_api.py --config_dir exp/bigvgan_teochew_24khz --checkpoint g_05130000 \
    --file_list test.txt --dataset_dir /data/ --output_dir results/ --max_samples 100

# CUDA kernel 加速 (CUDA kernel acceleration)
python bigvgan_api.py --config_dir exp/bigvgan_teochew_24khz --checkpoint g_05130000 \
    --input audio.wav --output out.wav --use_cuda_kernel

2. 作为 Python 包导入 (Python Package Import)

python

from bigvgan_standalone.bigvgan_api import BigVGANVocoder

# 加载 vocoder（config_dir 包含 config.json）
# Load vocoder (config_dir contains config.json)
vocoder = BigVGANVocoder(
    config_or_dir="exp/bigvgan_teochew_24khz",
    checkpoint_path="g_05130000"  # 文件名，自动拼接目录 / filename, auto-concatenated with dir
)

# 单文件重构 (Single file reconstruction)
vocoder.reconstruct_wav("input.wav", "output.wav")

# 批量重构 (Batch reconstruction)
vocoder.reconstruct_wav_batch(
    ["a.wav", "b.wav"],
    output_dir="results/"
)

# 从文件列表 (From file list)
vocoder.reconstruct_from_file_list(
    "filelists/teochew/test.txt",
    output_dir="results/",
    dataset_dir="/data/teochew_extla/",
    max_samples=100
)

# 从梅尔频谱文件 (From mel-spectrogram file)
vocoder.mel_file_to_wav("mel.npy", "output.wav")

# 底层 Tensor 级别操作 (Low-level Tensor operations)
audio = vocoder.load_wav("input.wav")    # auto-resample + normalize
mel = vocoder.compute_mel(audio)          # [1, 80, T]
recon = vocoder.mel_to_wav(mel)           # int16 np array

3. 在 Gradio / HuggingFace Space 中使用 (Use with Gradio / HuggingFace Space)

python

import gradio as gr
from bigvgan_standalone.bigvgan_api import BigVGANVocoder

vocoder = BigVGANVocoder("exp/bigvgan_teochew_24khz", "g_05130000")

demo = gr.Interface(
    fn=vocoder.gradio_reconstruct,
    inputs=gr.Audio(type="numpy"),
    outputs=gr.Audio(type="numpy"),
    title="BigVGAN Vocoder Demo"
)
demo.launch()

评估结果 (Evaluation Results)

使用 DNSMOS 指标对不同检查点进行评估 (DNSMOS metrics for different checkpoints):

Checkpoint	SIG	BAK	OVRL	Notes
g_05130000	3.4614	3.8736	3.1038	Best
g_05385000	3.4690	3.8536	3.0995	-
g_05215000	3.4671	3.8522	3.0964	-
g_05290000	3.4676	3.8491	3.0957	-
g_05325000	3.4658	3.8511	3.0952	-
g_05155000	3.4660	3.8489	3.0942	-
g_05310000	3.4656	3.8465	3.0929	-
g_05305000	3.4600	3.8575	3.0927	-
g_05285000	3.4645	3.8497	3.0924	-
g_05375000	3.4611	3.8499	3.0920	-
...	...	...	...	...

完整评估结果 (Full evaluation results, 80 checkpoints tested):

最佳性能出现在 g_05130000 (iteration 5,130,000)，OVRL 分数达到 3.1038，与 ground truth (3.1040) 非常接近。

(Best performance achieved at g_05130000 (iteration 5,130,000) with OVRL score of 3.1038, very close to ground truth at 3.1040.)

推荐使用 (Recommended): Checkpoint g_05130000，在 OVRL（整体质量 / overall quality）指标上表现最佳 (achieves the best score)。

性能对比 (Performance Comparison)

与同数据集训练的 HiFi-GAN 相比 (Compared with HiFi-GAN trained on the same dataset):

Model	DNSMOS OVRL	Training Time	Notes
BigVGAN (24khz)	3.1038	~10 days	Fine-tuned from pretrained
BigVGAN (22khz)	3.1030	~10 days	Fine-tuned from pretrained
HiFi-GAN	3.0724	~8 days	Trained from scratch
Ground Truth	3.1040	-	-

BigVGAN 在音质上略优于 HiFi-GAN (+0.0314)，几乎达到原始音频质量。

(BigVGAN slightly outperforms HiFi-GAN (+0.0314), nearly matching the original audio quality.)

panlr/BigVGAN_24khz_teochew

license: Apache License 2.0

BigVGAN (24khz) Vocoder for Teochew TTS

模型简介 (Model Description)

预训练权重 (Pre-trained Weights)

训练数据 (Training Data)

训练配置 (Training Configuration)

模型架构 (Model Architecture)

使用方法 (Usage)

1. 命令行界面 (Command Line Interface)

2. 作为 Python 包导入 (Python Package Import)

3. 在 Gradio / HuggingFace Space 中使用 (Use with Gradio / HuggingFace Space)

评估结果 (Evaluation Results)

性能对比 (Performance Comparison)

panlr/
BigVGAN_24khz_teochew