the original model can be found
hereWhisper is a pre-trained model for automatic speech recognition (ASR) and speech translation.
More details about it are available
here.
whisper-large-v2-tuned is a version of whisper-large-v2, fine-tuned by
ivrit.ai to improve Hebrew ASR using crowd-sourced labeling.
This model comes as a single checkpoint, whisper-large-v2-tuned.
It is a 1550M parameters multi-lingual ASR solution.
1from faster_whisper import WhisperModel
2
3model = WhisperModel("sivan22/faster-whisper-ivrit-ai-whisper-large-v2-tuned")
4
5segments, info = model.transcribe("audio.mp3")
6for segment in segments:
7 print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
You can use the
evaluate_model.py reference on GitHub to evalute the model's quality.
1@misc{marmor2023ivritai,
2 title={ivrit.ai: A Comprehensive Dataset of Hebrew Speech for AI Research and Development},
3 author={Yanir Marmor and Kinneret Misgav and Yair Lifshitz},
4 year={2023},
5 eprint={2307.08720},
6 archivePrefix={arXiv},
7 primaryClass={eess.AS}
8}
1@misc{radford2022whisper,
2 doi = {10.48550/ARXIV.2212.04356},
3 url = {https://arxiv.org/abs/2212.04356},
4 author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
5 title = {Robust Speech Recognition via Large-Scale Weak Supervision},
6 publisher = {arXiv},
7 year = {2022},
8 copyright = {arXiv.org perpetual, non-exclusive license}
9}
