khmyznikov/

whisper-int8-cpu-ort.onnx

Models was converted for this Subtitle Generator sample: https://github.com/microsoft/Windows-DevRel/tree/main/Samples/SubtitleGenerator

Steps to get these models manually:

1. Clone the Olive repository and navigate to the Whisper example folder:

git clone https://github.com/microsoft/Olive
cd Olive/examples/whisper

1. Install the required packages:

pip install olive-ai
python -m pip install -r requirements.txt
pip install onnxruntime==1.18.0
pip install onnxruntime_extensions

1. Prepare the Whisper model

1. Run the Olive workflow to generate the optimized model

olive run --config whisper_cpu_int8.json --setup
olive run --config whisper_cpu_int8.json

1. The generated model will be in the \models\conversion-transformers_optimization-onnx_dynamic_quantization-insert_beam_search-prepost folder.