ft-wmt14-5 – AI Model by lilferrit | AlphaNeural AI

You can deploy this model and start earning money today!

lilferrit/
ft-wmt14-5

like

0

text2text-generation

generated_from_trainer

lilferrit/wmt14-short

google/mt5-small

endpoints_compatible

Views

No views yet

ft-wmt14-5

This model is a fine-tuned version of google/mt5-small on the lilferrit/wmt14-short dataset. It achieves the following results on the evaluation set:

Loss: 2.0604
Bleu: 20.7584
Gen Len: 30.499

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 16
optimizer: Adafactor
lr_scheduler_type: constant
training_steps: 100000

Training results

Training Loss	Epoch	Step	Bleu	Gen Len	Validation Loss
1.9166	0.2778	10000	15.8119	32.097	2.3105
1.7184	0.5556	20000	17.5903	31.1153	2.1993
1.6061	0.8333	30000	18.9604	30.327	2.1380
1.516	1.1111	40000	19.1444	30.2727	2.1366
1.4675	1.3889	50000	19.7588	30.1127	2.1208
1.4416	1.6667	60000	19.9263	30.4463	2.0889
1.4111	1.9444	70000	2.0795	20.3323	30.1207
1.3603	2.2222	80000	2.0850	20.5373	30.5943
1.3378	2.5	90000	2.0604	20.7584	30.499
1.3381	2.7778	100000	2.0597	20.6113	30.701

Framework versions

Transformers 4.40.0
Pytorch 2.2.2+cu121
Datasets 2.19.0
Tokenizers 0.19.1