You can deploy this model and start earning money today!

LyngualLabs/
yecs-asr-llm-lid

Name: yecs-asr-llm-lid
Availability: InStock

fairseq2

automatic-speech-recognition

language-identification

Views

No views yet

YECS ASR — LLM with Inline Language-Tag Injection (yo/en)

A Yoruba–English code-switching ASR model that jointly transcribes speech and labels the language of every word inline:

<en> customer </en> <yo> sọ pé wọ́n fẹ́ kí </yo> <en> invoice </en> <yo> jẹ́ </yo> <en> separated </en>

It is omniASR_LLM_300M (Meta Omnilingual ASR — wav2vec2 encoder + LLaMA decoder, 1.63B params) fine-tuned on the YECS corpus, with four atomic tag tokens (<yo> </yo> <en> </en>) added to the vocabulary (9812 → 9816) and both the decoder embedding and output head resized. One model does transcription and per-word LID.

Result

Evaluated on the full held-out 9,949-utterance YECS test set with identical normalization (tags stripped from refs and hyps before WER):

Metric	LLM tag-injection	LLM plain†
WER (tone-aware)	16.27%	16.51%
WER (tone-insensitive)	13.64%	13.89%
CER (tone-aware)	6.27%	6.34%
Per-word LID accuracy	99.55%	—
Per-word LID macro-F1	99.55%	—

Tag injection slightly improves the LLM's WER (16.51 → 16.27, word-prefix 16.18) while adding 99.55% per-word language identification at zero extra inference cost. For reference, the CTC tag-injection variant (LyngualLabs/yecs-asr-ctc-lid) reaches 32.45% WER / 98.04% LID — lighter and faster, lower accuracy.

Usage

fairseq2 checkpoint (not 🤗 Transformers). Load with the omnilingual-asr stack via the included asset cards; pass lang="yor_Latn" for the LLM's language conditioning.

python

from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline
import torch
pipe = ASRInferencePipeline(model_card="omniASR_LLM_300M_lid_ft",
                            device=torch.device("cuda"), dtype=torch.bfloat16)
print(pipe.transcribe(["utt.wav"], lang=["yor_Latn"], batch_size=8))

Training

Base omniASR_LLM_300M; 5 epochs (7,750 steps), lr 1e-5, bf16, fsdp, grad-accum 4, 1×H200.
Embedding surgery keeps the special-token row (shifted) and inserts 4 tag rows; head appends 4 rows. Card sets both wav2vec2_asr_config.target_vocab_size and llama_config.vocab_size to 9816.
Normalization: NFC + lowercase + Unicode P/S strip; tone-aware keeps tone diacritics, tone-insensitive strips acute/grave/macron but keeps the phonemic under-dot.
W&B: afroscale_ai_cmu_africa/yecs-lid (run llm-tag-inject).
Code: https://github.com/osinkolu/yecs-asr-benchmark (tag-injection/).

Evaluation note

Scored on the full held-out YECS test split (9,949 utts); test audio is the canonical Mozilla/MDC release. Test reference labels are held privately by LyngualLabs (public test_metadata.csv has targets stripped), so metrics are internal and not independently reproducible. †LLM-plain (LyngualLabs/yecsASR-omni-llm, 16.51) is a separately-trained finetune re-scored here on the same 9,949/pipeline — comparable metric, but not the identical 5-epoch recipe as this tag model. (Its earlier reported 16.15% used a different normalization.)

License / data

Weights derive from Meta Omnilingual ASR; YECS corpus terms apply to the training data. Research use on Yoruba–English code-switching ASR + LID.

LyngualLabs/yecs-asr-llm-lid