You can deploy this model and start earning money today!

jensjepsen/
eo-mt-v13-large-bidir

Name: eo-mt-v13-large-bidir
Rating: 2.5 (8 reviews)

Views

No views yet

eo-mt-v13-large-bidir

MarianMT bidirectional English<->Esperanto, ~285M params (mt/configs/large.yaml). Trained on the v13 corpus, which drops the OPUS-100 KDE/GNOME/Ubuntu contamination that caused the v12 UI-label collapse failure mode (@ info: whatsthis -> & Resize, Name -> A tool to store a new image).

Config (large.yaml)

d_model 1024, 8 enc + 8 dec layers, 16 heads, ffn 4096. Total: 285.3M / trainable 284.3M params.

Data

jensjepsen/esperanto-mt-parallel-v13 (5.03M rows; no opus-100)
jensjepsen/esperanto-mt-math-parallel-v2 (sentences + rows)
jensjepsen/esperanto-mt-yago-parallel-v2 (labels + comments)
jensjepsen/esperanto-orca-math-gemini-parallel

Total pool: 5.84M rows/epoch.

Training

Single RTX 5090, bf16, 3 epochs (136,962 steps), effective batch 128 (per-device 64 x grad_accum 2), peak LR 2e-4, cosine decay, warmup 2000 steps, weight_decay 0.01, label smoothing 0.1, dropout 0.1, direction=bidir, val-direction=en2eo. Runtime ~10h 30m.

Tokenizer: spm_eneo_48k_v3.model (SPM unigram, case-preserving nmt_nfkc).

Final val BLEU (cased sacrebleu, ep 3.0)

set	v13 large	v12 large	Δ
FLORES devtest (en2eo)	28.37	28.26	+0.11
opus100 validation	34.02	33.99	+0.03
MMLU STEM	68.49	68.90	-0.41
MMLU full	61.18	60.58	+0.60
SciQ	66.18	66.85	-0.67
Math WP	99.70	99.69	+0.01

Beats v12 on 4/6 metrics. Peak FLORES BLEU 28.40 at ep 1.75 (best of any run to date on this eval).

Eyeball evaluation: preserves named-work titles bilingually (The Vampire Diaries (La Vampiro Diaries) estas usona televida serio de supernaturaj dramoj), keeps LaTeX formatting clean (v12 introduced spurious spaces around parens), handles distinctions v13-tiny loses (zip-kodo vs poshkodo), and produces more natural English on EO->EN (at a red light vs at red light).

Loading

python

from transformers import MarianMTModel
import sentencepiece as spm

model = MarianMTModel.from_pretrained("jensjepsen/eo-mt-v13-large-bidir")
sp = spm.SentencePieceProcessor(model_file="spm_eneo_48k_v3.model")
# Prepend <eo> (id=48001) to input for EN->EO, or <en> (id=48000) for EO->EN.