This is
whisper-medium model converted to the
OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by
NNCF .
For more information on quantization, check the
OpenVINO model optimization guide .
from datasets import load_dataset
from transformers import AutoProcessor
from optimum.intel.openvino import OVModelForSpeechSeq2Seq
model_id = "OpenVINO/whisper-medium-int4-ov"
tokenizer = AutoProcessor.from_pretrained(model_id)
model = OVModelForSpeechSeq2Seq.from_pretrained(model_id)
dataset = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation", trust_remote_code=True)
sample = dataset[0]
input_features = tokenizer(
sample["audio"]["array"],
sampling_rate=sample["audio"]["sampling_rate"],
return_tensors="pt",
).input_features
outputs = model.generate(input_features)
text = tokenizer.batch_decode(outputs)[0]
print(text)pip install huggingface_hub "datasets<4" librosa soundfile
pip install -U --pre --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly openvino openvino-tokenizers openvino-genaiimport huggingface_hub as hf_hub
model_id = "OpenVINO/whisper-medium-int4-ov"
model_path = "whisper-medium-int4-ov"
hf_hub.snapshot_download(model_id, local_dir=model_path)
import openvino_genai as ov_genai
import datasets
device = "CPU"
pipe = ov_genai.WhisperPipeline(model_path, device)
dataset = datasets.load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation", trust_remote_code=True)
sample = dataset[0]["audio"]["array"]
print(pipe.generate(sample))More GenAI usage examples can be found in OpenVINO GenAI library
docs and
samples Check the original model card for
original model card for limitations.
The original model is distributed under
apache-2.0 license. More details can be found in
original model card .
Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See
Intel’s Global Human Rights Principles . Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.
