A fine-tune of google/byt5-small that restores Yorùbá diacritics (tone marks: ◌́ high, ◌̀ low, unmarked mid; underdots: ẹ ọ ṣ) to undiacritised text. A byte-level model suits the task because the target is a byte-level phenomenon: a combining mark on a base character.

Preprocessing for TTS and MT, query normalisation, and corpus annotation. Ambiguous forms are resolved from context (owo → owó money / ọwọ́ hand / ọ̀wọ̀ honour). Byte sequences are long, so chunk inputs beyond ~1024 bytes. Training data is Yorùbá news and scripture, so conversational, social-media, and dialectal Yorùbá are out of distribution. Greedy decoding (num_beams=1) is a good default: the task is near-monotonic, so greedy is fast and accurate. Beam search (num_beams=5) is comparable in quality, but only on transformers<5 (verified on 4.57.6): transformers 5.0.0 has a beam-search regression that truncates this byte-level model's output.

Yorùbá diacritics have multiple valid encodings that render identically but differ byte-for-byte. Normalise both hypothesis and reference to the same form (unicodedata.normalize("NFC", ...)) before scoring. Skipping this can shift CER by tens of points.

The Bible text (Bíbélì Mímọ́ ní Èdè Yorùbá Òde-Òní, Biblica, Inc.) is segmented below verse level on punctuation, so it does not align sentence-for-sentence with other published Yorùbá Bible splits. JW300 was excluded because its Yorùbá diacritisation is inconsistent.

Audited against YAD dev+test (the MENYO-20k test split). The only overlap is one boilerplate header line, itself duplicated within MENYO-20k's own train/test halves and therefore also present in YAD's official train split. The model can be evaluated on YAD test.

Fine-tuned from google/byt5-small on Google Cloud TPU v6e-8 in two phases. Both phases use per-device batch 4, gradient accumulation 2 (effective batch 64), AdamW, linear decay, weight decay 0.01, and gradient clipping 0.5. Phase 1 uses learning rate 2e-4 with 300 warmup steps. Phase 2, which produced the released checkpoint, uses learning rate 1e-4 with no warmup and runs 4 epochs. Frameworks: Transformers 4.53.3, PyTorch 2.6.0+cu124, torch_xla.

Greedy decoding (num_beams=1), both sides normalised to the same Unicode form. Two test sets are reported and are not interchangeable: YAD is the public benchmark and the only set comparable to published work; the internal test set is reported for continuity with earlier versions of this card and is comparable to nothing else.

Each is scored against two references. Official is the reference as released, the basis of every published YAD number and the column to quote alongside prior work. Corrected repairs the underdot encoding described below on both sides.

The two columns answer different questions rather than one being a discount on the other. Under the official reference, DER counts only positions where the reference carries a mark the metric recognises, so the 891 underdot positions encoded as U+0329 are dropped from the denominator entirely and the model's output there is never examined. Correcting the reference returns those positions to the count. WER and CER can only improve under correction; DER and WDER may move in either direction, because correction enlarges their denominator, which raises the rate if the model handles the recovered positions worse than average.

† Custom metrics carried over from earlier versions of this card. They perform no encoding repair, so they have an official-reference value only. BLEU and ChrF are computed on raw strings and so do not vary by reference.

All numbers were measured with transformers 4.57.6 and greedy decoding as described above; the YAD figures reproduced identically across three independent evaluation runs.

DER is diacritic-position error (wrong marks ÷ marks that should be present; a no-diacritic output scores 100%), the convention published YAD results use. The YAD reference encodes 1,415 of 138,798 positions (1.02%) with the wrong underdot codepoint (U+0329 for U+0323) or doubled underdots, which normalisation cannot fix.

DER (underdot) against the official reference does not measure those positions at all. The metric takes an underdot to be U+0323, so a reference position carrying U+0329 matches nothing it looks for and is skipped: 891 positions, 2.65% of the benchmark's real underdots, absent from the denominator. That column therefore reports the underdot error rate over the subset of underdots YAD happened to encode correctly. The corrected column measures all of them, and is the one to cite for any claim about underdot behaviour.

Weights are Apache 2.0, matching google/byt5-small. Corpus licences govern the training data and require attribution: MENYO-20k is CC BY-NC 4.0 (Adelani et al., 2021); the Biblica® Bíbélì Mímọ́ ní Èdè Yorùbá Òde-Òní (2017) is CC BY-SA (Biblica, Inc.).

Trained as a member of the HausaNLP Research Group using Google TPU Research Cloud (TRC) compute.