A "back of the envelope" evaluation comparing fine-tuned Whisper models running locally against commercial ASR APIs via Eden AI.
Can fine-tuning Whisper achieve measurable WER reductions, even when comparing local inference against cloud-based commercial models?
Yes. Fine-tuned Whisper Large Turbo running locally achieved 5.84% WER, beating the best commercial API (Assembly at… See the full description on the dataset page:
https://huggingface.co/datasets/danielrosehill/Whisper-Fine-Tune-One-Shot-Eval.
