Views
No views yet
| Benchmark | chrF | chrF++ | BLEU |
|---|---|---|---|
| gold (human references, article-level, N=500) | 46.89 | 38.89 | 3.25 |
| held-out chunk (in-distribution) | 67.74 | 60.74 | 21.34 |
| held-out sentence (in-distribution) | 64.41 | 57.76 | 19.91 |
Input (en): While this is a 7.6 percent increase compared to the same period last year, an average of 7,778 tourists visit the Maldives daily.Output (dv): މިއީ މިދިޔަ އަހަރުގެ މި މުއްދަތާ އަޅާބަލާއިރު 7.6 އިންސައްތައިގެ ކުރިއެރުމެއް ކަމަށްވާއިރު، ދުވާލަކު އެވްރެޖްކޮށް 7،778 ފަތުރުވެރިން ރާއްޖެ ޒިޔާރަތްކުރެއެވެ.
1import torch
2from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
3
4m = "Neobe/en-dhivehi-byt5-large-sentence"
5tok = AutoTokenizer.from_pretrained(m)
6model = AutoModelForSeq2SeqLM.from_pretrained(m, torch_dtype=torch.float32).eval().cuda() # fp32
7
8src = "The President of the Maldives met with the cabinet today."
9inp = tok(src, return_tensors="pt", truncation=True, max_length=1024).to("cuda")
10out = model.generate(**inp, max_new_tokens=512, num_beams=4, length_penalty=1.0, no_repeat_ngram_size=0)
11print(tok.decode(out[0], skip_special_tokens=True))google/byt5-large; fp32; Adafactor; LR 1e-4 cosine; max_length 1024; 1 epoch; effective batch ~32; gradient checkpointing.1@misc{neobe_en_dhivehi_byt5_large_sentence_2026,
2 title = {byT5-large English→Dhivehi (sentence-level)},
3 author = {Neobe},
4 year = {2026},
5 howpublished = {\url{https://huggingface.co/Neobe/en-dhivehi-byt5-large-sentence}}
6}