Views
No views yet
1from transformers import AutoTokenizer, T5ForConditionalGeneration
2
3tok = AutoTokenizer.from_pretrained("ghananlpcommunity/ghana-english-ipa2text-tokenizer")
4model = T5ForConditionalGeneration.from_pretrained("ghananlpcommunity/t5-small-ipa2text-v2")
5
6ipa = "ɡ ʊ d m ɔː n ɪ ŋ , l e d i s æ n d dʒ ɛ n t ə l m ɛ n !"
7ids = tok(ipa, return_tensors="pt").input_ids
8out = model.generate(ids)
9print(tok.decode(out[0], skip_special_tokens=True))
10# "Good morning, ladies and gentlemen!"ghananlpcommunity/ghana-english-ipa2text-tokenizer),
not stock T5-small's — this model's vocabulary was extended with phone tokens it needs.scripts/infer.py in this repo, or the
poto-asr library.ghananlpcommunity/ghana-gov-speeches-sentences-ipa-robust:
Ghana government/parliamentary speech, phonemised with
ghana-english-g2p. Training input isn't clean
G2P output as-is — each batch, most examples are run through one of five random perturbations
(vowel-length collapse, weak-vowel deletion, consonant confusion/deletion, phone insertion) that
mimic real ASR phoneme-recognizer errors, so the model has seen noisy input, not just perfect
phonemes.generation_config.json already sets no_repeat_ngram_size=3 and
repetition_penalty=1.3 to guard against it — don't override those unless you've checked the
effect on your inputs.