Views
No views yet
facebook/wav2vec2-large-960h and fine tuned it using 1400 audio clips (around 10-15 seconds each) from various cryptocurrency related podcasts. To label the data, we downloaded cryptocurrency podcasts from youtube with their subtitle data and split the clips up by sentence. We then compared the youtube transcription with facebook/wav2vec2-large-960h to correct many mistakes in the youtube transcriptions. We can probably achieve better results with more data clean up.facebook/wav2vec2-large-960h only reached a WER of 27% on our data.1from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
2from datasets import load_dataset
3import soundfile as sf
4import torch
5
6
7# load model and tokenizer
8processor = Wav2Vec2Processor.from_pretrained("distractedm1nd/wav2vec-en-finetuned-on-cryptocurrency")
9model = Wav2Vec2ForCTC.from_pretrained("distractedm1nd/wav2vec-en-finetuned-on-cryptocurrency"
10
11filename = "INSERT_FILENAME"
12audio, sampling_rate = sf.read(filename)
13
14input_values = processor(audio, return_tensors="pt", padding="longest", sampling_rate=sampling_rate).input_values # Batch size 1
15
16
17# retrieve logits
18logits = model(input_values).logits
19
20# take argmax and decode
21predicted_ids = torch.argmax(logits, dim=-1)
22tokenizer.batch_decode(predicted_ids