Preprocessing
- Normalisation and tokenisation with moses scripts
- truecased with model docgWP.tcmodel.[LAN] and moses scripts
- bped with model model.caesen40k.bpe and subword-nmt
- Note: no prepended tag for multilinguality
Training Data
- Bilingual es-ca: DOGC, Wikimatrix, OpenSubtitles, JW300, GlobalVoices
- Bilingual es-ca: Translations using systems trained with 1. of Oscar and Wikipedia
- Bilingual es-en, ca-en: United Nations, Europarl, Wikimatrix, OpenSubtitles, JW300
- Bilingual es-en, ca-en: Translations using systems trained with 1. of the missing pairs
- Final training data size for the ca/es-en: 44M parallel sentences
- Finetuned with 1.5M real parallel data (without backtranslations)
Model
Transformer big with guided alignments. Relevant parameters:
--beam-size 6
--normalize 0.6
--enc-depth 6 --dec-depth 6 --transformer-heads 8
--transformer-preprocess n --transformer-postprocess da
--transformer-dropout 0.1
--label-smoothing 0.1
--dim-emb 1024 --transformer-dim-ffn 4096
--transformer-dropout-attention 0.1
--transformer-dropout-ffn 0.1
--learn-rate 0.00015 --lr-warmup 8000 --lr-decay-inv-sqrt 8000
--optimizer-params 0.9 0.998 1e-09
--clip-norm 5
--tied-embeddings
--exponential-smoothing
--transformer-guided-alignment-layer 1 --guided-alignment-cost mse --guided-alignment-weight 0.1
Evaluation
Test set
ca2en
BLEU|#:1|bs:1000|rs:12345|c:mixed|e:no|tok:13a|s:exp|v:2.0.0 = 47.8 (μ = 47.8 ± 0.9)
chrF|#:1|bs:1000|rs:12345|c:mixed|e:yes|nc:6|nw:0|s:no|v:2.0.0 = 69.9 (μ = 69.9 ± 0.7)
es2en
BLEU|#:1|bs:1000|rs:12345|c:mixed|e:no|tok:13a|s:exp|v:2.0.0 = 48.9 (μ = 48.9 ± 0.9)
chrF2|#:1|bs:1000|rs:12345|c:mixed|e:yes|nc:6|nw:0|s:no|v:2.0.0 = 70.5 (μ = 70.5 ± 0.7)
