Parameter	Value
Source model	`openai/whisper-small`
Weight format	INT8 asymmetric (per-channel)
Group size	128
Encoder layers compressed	74 / 74 (100%)
Decoder layers compressed	122 / 122 (100%)
Task	`automatic-speech-recognition-with-past`

Toolchain

Package	Version
Python	3.11.9
openvino	2026.0.0
openvino-genai	2026.0.0.0
openvino-tokenizers	2026.0.0.0
optimum-intel	1.27.0
optimum	2.1.0
nncf	3.0.0
transformers	4.57.6
torch	2.11.0

Usage

python

import numpy as np
import openvino_genai as ov_genai

pipe = ov_genai.WhisperPipeline("ov-whisper_small-int8-2026.0.0", "CPU")

# Load audio as 16 kHz float32 mono (e.g. via librosa)
import librosa
samples, _ = librosa.load("audio.wav", sr=16000, mono=True)
samples = np.asarray(samples, dtype=np.float32)

result = pipe.generate(samples)
print(result.text)

Supported devices: CPU, GPU, NPU (tested on Intel Core Ultra 7 255H / Arc 140T / AI Boost).

Reproduce the export

bash

pip install -r requirements.txt
python export_whisper_int8_ov.py \
    --model openai/whisper-small \
    --output ov-whisper_small-int8-2026.0.0 \
    --cache-dir ./cache_dir

Or equivalently with optimum-cli directly:

bash

optimum-cli export openvino \
    -m openai/whisper-small \
    --task automatic-speech-recognition-with-past \
    --weight-format int8 \
    --group-size 128 \
    ov-whisper_small-int8-2026.0.0

Validate

python validate_whisper_genai.py ov-whisper_small-int8-2026.0.0 --device CPU

Files

openvino_encoder_model.bin/.xml -- Whisper encoder (INT8)
openvino_decoder_model.bin/.xml -- Whisper decoder with past/beam_idx (INT8)
openvino_tokenizer.bin/.xml -- Tokenizer
openvino_detokenizer.bin/.xml -- Detokenizer
config.json, generation_config.json -- Model configuration
tokenizer.json, vocab.json, merges.txt -- Tokenizer data
export_whisper_int8_ov.py -- Export script used to produce this model
validate_whisper_genai.py -- Smoke-test script
requirements.txt -- Pinned Python dependencies

hlevring/ov-whisper_small-int8-2026.0.0

ov-whisper_small-int8-2026.0.0

Quantization details

Toolchain

Usage

Reproduce the export

Validate

Files

hlevring/
ov-whisper_small-int8-2026.0.0