You can deploy this model and start earning money today!

ghananlpcommunity/
w2v-bert-2.0_twi_alpha_v1-onnx-int8

Name: w2v-bert-2.0_twi_alpha_v1-onnx-int8
Availability: InStock

Views

No views yet

ghananlpcommunity/w2v-bert-2.0_twi_alpha_v1-onnx-int8

ONNX dynamic int8 quantized version of ghananlpcommunity/w2v-bert-2.0_twi_alpha_v1. Exported via torch.onnx (optimum does not yet support wav2vec2-bert natively).

How to load

python

import onnxruntime as ort
from transformers import AutoProcessor
import numpy as np

processor = AutoProcessor.from_pretrained("ghananlpcommunity/w2v-bert-2.0_twi_alpha_v1-onnx-int8")
session = ort.InferenceSession(
    "model_quantized_int8.onnx",
    providers=["CPUExecutionProvider"],
)

# audio_array: numpy float32 at 16kHz
inputs = processor(audio_array, sampling_rate=16000, return_tensors="np")
logits = session.run(["logits"], {
    "input_features": inputs["input_features"],
    "attention_mask": inputs["attention_mask"],
})[0]

predicted_ids = np.argmax(logits, axis=-1)
transcription = processor.batch_decode(predicted_ids)[0]

Requirements

pip install onnxruntime transformers

Performance vs fp32

~60% smaller on disk
~2–4x faster on CPU