You can deploy this model and start earning money today!

ghananlpcommunity/
w2v-bert-2.0_twi_alpha_v1-pytorch-int8

Name: w2v-bert-2.0_twi_alpha_v1-pytorch-int8
Availability: InStock

Views

No views yet

ghananlpcommunity/w2v-bert-2.0_twi_alpha_v1-pytorch-int8

PyTorch dynamic int8 quantized version of ghananlpcommunity/w2v-bert-2.0_twi_alpha_v1.

How to load

python

from transformers import AutoProcessor, Wav2Vec2BertForCTC
import torch

processor = AutoProcessor.from_pretrained("ghananlpcommunity/w2v-bert-2.0_twi_alpha_v1-pytorch-int8")
model = Wav2Vec2BertForCTC.from_pretrained("ghananlpcommunity/w2v-bert-2.0_twi_alpha_v1-pytorch-int8", torch_dtype=torch.float32)
model = torch.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8)

state_dict = torch.hub.load_state_dict_from_url(
    f"https://huggingface.co/ghananlpcommunity/w2v-bert-2.0_twi_alpha_v1-pytorch-int8/resolve/main/quantized_model_int8.pt",
    map_location="cpu",
)
model.load_state_dict(state_dict)
model.eval()

Performance

~40% smaller on disk than fp32
~1.5–2x faster on CPU
CER / WER unchanged (lossless for activations, lossy only for weights)