nllb-en-es
This model is a fine-tuned version of
facebook/nllb-200-distilled-600M on an unknown dataset.
It achieves the following results on the evaluation set:
Model description
This model is the first iteration of a larger fine-tuning series I'm developing for a NLP Pipeline that handles translations between English,
Spanish, Amharic, and Uzbek. The base model is Facebook's No Language Left Behind, and was chosen for its inclusion of low-resource languages.
Intended uses & limitations
Text to Text Translation between English --> Spanish
The intended use of this model is to handle bidirectional English-centric translation, but so far only one direction for a single pair has been developed.
Training procedure
Trained on Google ColabPRO using an OPUS Books dataset, with High RAM and A100 GPU.
Training hyperparameters
The following hyperparameters were used during training:
- Transformers 5.0.0
- Pytorch 2.10.0+cu128
- Datasets 4.0.0
- Tokenizers 0.22.2
More hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 1
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|
| 1.8802 | 1.0 | 18694 | 1.7770 |
Framework versions
- Transformers 5.0.0
- Pytorch 2.10.0+cu128
- Datasets 4.0.0
- Tokenizers 0.22.2
