This model is a fine-tune of the NLLB-200 3.3B dense variant developed by Meta AI, adapted specifically for the English–Catalan translation direction. NLLB-200 (No Language Left Behind) is a multilingual encoder-decoder Transformer capable of translating between 200 languages, with a particular focus on low-resource languages.

Developed by: Francisco Belenguer
Model type: Encoder-decoder Transformer (M2M-100 architecture)
Languages: English (eng_Latn) → Catalan (cat_Latn)
License: CC-BY-NC 4.0 (inherited from base model)
Finetuned from: facebook/nllb-200-3.3B

Model Architecture

The base model is a dense Transformer encoder-decoder with the following configuration:

Parameter	Value
Total parameters	~3.3B
Encoder layers	24
Decoder layers	24
Attention heads	16
FFN dimension	8192
Hidden size (d_model)	1024
Vocabulary size	256,206
Max input length	512 tokens

Model Sources

Base model repository: facebook/nllb-200-3.3B
Base model paper: NLLB Team et al., No Language Left Behind: Scaling Human-Centered Machine Translation, arXiv, 2022

Uses

Direct Use

Single-sentence translation from English to Catalan. Suitable for research and experimentation in machine translation.

python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import torch

model_id = "franciscobdl/nllb-200-3.3B-en-ca"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda")

text = "The weather is beautiful today."
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True).to("cuda")

with torch.no_grad():
    output = model.generate(
        **inputs,
        forced_bos_token_id=tokenizer.convert_tokens_to_ids("cat_Latn"),
        max_length=256,
        num_beams=4,
        early_stopping=True,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

Out-of-Scope Use

Production deployment: This model is not intended for production use.
Domain-specific texts: Not suitable for medical, legal, or other specialized domains.
Document translation: The model was trained on sentence-level data; document-level translation may degrade quality.
Sequences exceeding 512 tokens: Performance degrades significantly beyond this limit.
Certified translations: Outputs cannot be used as certified translations.

Evaluation

Testing Data

Evaluated on the FLORES-200 devtest split, which contains 1,012 professionally translated sentences covering a broad range of topics (Wikipedia-derived content).

Dataset: FLORES-200
Split: devtest
Source language: eng_Latn
Target language: cat_Latn

Metrics

Evaluation was performed using three standard machine translation metrics:

Metric	Description
BLEU (sacrebleu)	N-gram precision with brevity penalty. Standard lexical metric.
chrF	Character n-gram F-score. More robust to morphological variation.
COMET (wmt22-comet-da)	Neural metric based on multilingual embeddings. Correlates strongly with human judgement.

Results

Evaluated with num_beams=4, max_length=256, on FLORES-200 devtest (1,012 sentences):

BLEU	chrF	COMET
42.33	66.64	0.8767

Training Details

Training Data

The fine-tune was trained on projecte-aina/CA-EN_Parallel_Corpus, a Catalan–English parallel corpus created by the Language Technologies Unit at the Barcelona Supercomputing Center (BSC) as part of the Projecte AINA, funded by the Generalitat de Catalunya.

The dataset consists of parallel sentence pairs in Catalan and English from a mix of domains and styles, partially derived from web crawlings. Each sentence pair includes quality scores: language probability (via Lingua.py) and alignment score (via LaBSE). Sentence pairs below a LaBSE alignment threshold of 0.75 or a language probability below 0.5 were filtered out.

The base model (facebook/nllb-200-3.3B) was originally trained on parallel multilingual data from a variety of public sources detailed in the NLLB-200 paper (Section 5), supplemented with monolingual data from Common Crawl.

Training Procedure

Training Hyperparameters

Training regime: [More Information Needed]

Bias, Risks, and Limitations

The model has been primarily tested on Wikimedia-domain text (via FLORES-200). Performance on other domains (news, social media, technical content) is not guaranteed.
Catalan has regional variants; the model may not capture all of them equally.
Mistranslations may occur, particularly for rare vocabulary or complex syntax.
The model should not be used to translate content where accuracy is safety-critical.

Recommendations

Users should evaluate the model on their specific domain and use case before deployment. For texts longer than a few sentences, quality may degrade due to the 512-token input limit.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator.

Hardware Type: [More Information Needed]
Hours used: [More Information Needed]
Cloud Provider: [More Information Needed]
Compute Region: [More Information Needed]
Carbon Emitted: [More Information Needed]

Citation

If you use this model, please cite the original NLLB-200 paper:

bibtex

@article{nllb2022,
  title={No Language Left Behind: Scaling Human-Centered Machine Translation},
  author={NLLB Team and others},
  journal={arXiv preprint arXiv:2207.04672},
  year={2022}
}