Name: whisper-tiny-vi
Availability: InStock

You can deploy this model and start earning money today!

whisper tiny fine-tuned on a very big collection of vietnamese speech datasets

TODO:

training then publish checkpoint
evaluate WER on Common Voice & FLEURS & VIVOS
convert to openai-whisper, whisper.cpp, faster-whisper
convert to ONNX: to try https://github.com/k2-fsa/sherpa-onnx & https://github.com/zhuzilin/whisper-openvino
convert to TensorRT: https://github.com/openai/whisper/discussions/169

21k steps, warm-up 5%, batch size 16×2 (kaggle free T4×2)

manually evaluate WER on test set - vietnamese part:

@ `float16`	`CommonVoice v16.1`	`FLEURS`	`VIVOS`
original `whisper-tiny`	>100%	88.6%	62.5%
this model	26.6%	37.1%	18.7%

all training + evaluation scripts are on my repo: https://github.com/phineas-pta/fine-tune-whisper-vi

usage example:

python

import torch
from transformers import pipeline

PIPE = pipeline(task="automatic-speech-recognition", model="doof-ferb/whisper-tiny-vi", device="cuda:0", torch_dtype=torch.float16)
PIPE_KWARGS = {"language": "vi", "task": "transcribe"}

PIPE("audio.mp3", generate_kwargs=PIPE_KWARGS)["text"]