Views
No views yet
1from transformers import AutoModel
2import soundfile as sf
3import librosa
4
5# Initialize from the trained model
6model = AutoModel.from_pretrained(
7 "",
8 torch_dtype=torch.float16,
9 trust_remote_code=True
10)
11model.to("cuda")
12model.eval()
13
14# read a wav file (it needs to be in 16 kHz and clipped to 30 seconds)
15audio, sr = sf.read("path_to_your_audio.wav")
16if len(audio.shape) == 2:
17 audio = audio[:, 0]
18if len(audio) > 30 * sr:
19 audio = audio[: 30 * sr]
20if sr != 16000:
21 audio = librosa.resample(audio, orig_sr=sr, target_sr=16000, res_type="fft")
22
23# Run generation
24prompt_pattern="<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n<Speech><SpeechHere></Speech> {}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"
25response = model.generate(
26 audio=audio,
27 prompt="transcribe this audio",
28 prompt_pattern=prompt_pattern,
29 do_sample=False,
30 max_new_tokens=512,
31 repetition_penalty=1.1,
32 num_beams=1,
33 # temperature=0.4,
34 # top_p=0.9,
35)
36print(response)soundfile.read or librosa.resample to read a wav file like the example abovestr) -- Text input to the modelstr) -- Chat template that is augmented with special tokens, and it must be set the same as one during trainingint, optional, defaults to 1024)int, optional, defaults to 4)bool, optional, defaults to True)float, optional, defaults to 0.9)float, optional, defaults to 1.0),float, optional, defaults to 1.0),float, optional, defaults to 1.0),model.generate_stream() for streaming generation. Please refer to modeling_typhoonaudio.py for this function.