Base Model: google-t5/t5-small
Dataset Used: opus_tatoeba (French-English)
Fine-tuning Epochs: 3
Optimizer: AdamW (learning rate: 5e-5)
Evaluation Metric: BLEU Score
Fine-Tuned BLEU Score: 47.00 (on Tatoeba test set, 10% random subset of tatoeba)

Model Description

Developed by: Mahdi Ihdeme
Model type: Language model

Usage

python

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_name = "mihdeme/t5-fr-en-tatoeba"
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

def translate(sentence, max_length=460, num_beams=5):
    inputs = tokenizer(f"translate French to English: {sentence}", return_tensors="pt", padding=True, truncation=True)
    outputs = model.generate(
        **inputs,
        max_length=max_length,  
        num_beams=num_beams,
    )
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

print(translate("Bonjour, comment ça va ?"))

Training Configuration

Batch Size: 16
Max Sequence Length: 460
Hardware Used: Google Colab GPU (Tesla T4)

License

Apache 2.0

Acknowledgments

Trained using Hugging Face Transformers. Original dataset from Tatoeba.