Dataset: Mozilla Common Voice (Vietnamese subset)
Audio Preprocessing: 16kHz resampling, Gaussian noise injection, VAD-based silence trimming via Librosa
Augmentation: SpecAugment (frequency + time masking)
Optimizer: AdamW with linear warmup scheduling
Gradient Accumulation: 4 steps
Evaluation Metrics: Word Error Rate (WER), Character Error Rate (CER)

Audio Pipeline

Raw Audio → 16kHz Resampling → Silence Trimming (VAD)
         → Noise Augmentation → Log-Mel Spectrogram
         → SpecAugment → Whisper Encoder-Decoder

Usage

python

from transformers import WhisperProcessor, WhisperForConditionalGeneration
import librosa

model = WhisperForConditionalGeneration.from_pretrained("sanvo/vietnamese-whisper-asr")
processor = WhisperProcessor.from_pretrained("openai/whisper-small")

audio, sr = librosa.load("vietnamese_audio.wav", sr=16000)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
predicted_ids = model.generate(inputs.input_features)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Features

Vietnamese-optimized speech recognition
Microphone + file upload Gradio demo
WER/CER evaluation pipeline
Audio preprocessing with Librosa (resampling, noise augmentation, silence trimming)

sanvo/vietnamese-whisper-asr

Vietnamese Speech Recognition with Whisper

Model Details

Training

Audio Pipeline

Usage

Features

Links

sanvo/
vietnamese-whisper-asr