This model builds upon the capabilities of Whisper small (a pre-trained model for speech recognition and translation trained on a massive 680k hour dataset). While Whisper demonstrates impressive generalization abilities, this model takes it a step further to be very specific for Nigerian accents.
In this example, the context tokens are 'unforced', meaning the model automatically predicts the output language
(English) and task (transcribe).
1>>> from transformers import WhisperProcessor, WhisperForConditionalGeneration
2>>> from datasets import load_dataset
3
4>>> # load model and processor
5>>> processor = WhisperProcessor.from_pretrained("AWARRITech/naijaspeech-Whisper-small-en")
6>>> model = WhisperForConditionalGeneration.from_pretrained("AWARRITech/naijaspeech-Whisper-small-en")
7>>> model.config.forced_decoder_ids = None
8
9>>> # load dummy dataset and read audio files
10>>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
11>>> sample = ds[0]["audio"]
12>>> input_features = processor(sample["array"], sampling_rate=sample["sampling_rate"], return_tensors="pt").input_features
13
14>>> # generate token ids
15>>> predicted_ids = model.generate(input_features)
16>>> # decode token ids to text
17>>> transcription = processor.batch_decode(predicted_ids, skip_special_tokens=False)
18['<|startoftranscript|><|en|><|transcribe|><|notimestamps|> Mr. Quilter is the apostle of the middle classes and we are glad to welcome his gospel.<|endoftext|>']
19
20>>> transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
21[' Mr. Quilter is the apostle of the middle classes and we are glad to welcome his gospel.']
The context tokens can be removed from the start of the transcription by setting skip_special_tokens=True.
