You can deploy this model and start earning money today!

jamesngai/
xlm-roberta-xl-final-lora500

Name: xlm-roberta-xl-final-lora500
Availability: InStock

safetensors

generated_from_trainer

facebook/xlm-roberta-xl

finetune

mit

Views

No views yet

xlm-roberta-xl-final-lora500

This model is a fine-tuned version of facebook/xlm-roberta-xl on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.5378
Precision: 0.9334
Recall: 0.9341
F1: 0.9337
Accuracy: 0.9421

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 16
eval_batch_size: 16
seed: 42
distributed_type: multi-GPU
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 40
num_epochs: 40
mixed_precision_training: Native AMP
label_smoothing_factor: 0.2

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
2.6949	1.0	250	1.9846	0.7571	0.8198	0.7872	0.8265
1.8141	2.0	500	1.6856	0.8709	0.8824	0.8766	0.8938
1.6277	3.0	750	1.6081	0.8881	0.9011	0.8945	0.9122
1.5464	4.0	1000	1.5735	0.9004	0.9064	0.9034	0.9201
1.4908	5.0	1250	1.5482	0.9111	0.9145	0.9128	0.9274
1.4599	6.0	1500	1.5386	0.9096	0.9175	0.9135	0.9282
1.4382	7.0	1750	1.5396	0.9175	0.9204	0.9189	0.9292
1.422	8.0	2000	1.5394	0.9163	0.9212	0.9188	0.9305
1.4053	9.0	2250	1.5354	0.9240	0.9223	0.9231	0.9335
1.3949	10.0	2500	1.5424	0.9155	0.9230	0.9192	0.9308
1.3858	11.0	2750	1.5405	0.9202	0.9248	0.9225	0.9313
1.379	12.0	3000	1.5364	0.9186	0.9263	0.9224	0.9339
1.3715	13.0	3250	1.5310	0.9263	0.9275	0.9269	0.9373
1.3647	14.0	3500	1.5321	0.9221	0.9273	0.9247	0.9355
1.3592	15.0	3750	1.5347	0.9277	0.9261	0.9269	0.9372
1.3564	16.0	4000	1.5323	0.9229	0.9269	0.9249	0.9371
1.3524	17.0	4250	1.5339	0.9232	0.9248	0.9240	0.9347
1.3512	18.0	4500	1.5425	0.9262	0.9284	0.9273	0.9370
1.3482	19.0	4750	1.5387	0.9238	0.9299	0.9268	0.9362
1.3437	20.0	5000	1.5334	0.9267	0.9324	0.9295	0.9389
1.3414	21.0	5250	1.5379	0.9302	0.9283	0.9292	0.9394
1.3408	22.0	5500	1.5394	0.9256	0.9291	0.9273	0.9381
1.3401	23.0	5750	1.5376	0.9320	0.9301	0.9310	0.9391
1.3388	24.0	6000	1.5381	0.9300	0.9300	0.9300	0.9383
1.3379	25.0	6250	1.5402	0.9247	0.9309	0.9278	0.9380
1.3361	26.0	6500	1.5415	0.9303	0.9275	0.9289	0.9383
1.3349	27.0	6750	1.5391	0.9305	0.9300	0.9302	0.9402
1.3338	28.0	7000	1.5379	0.9296	0.9290	0.9293	0.9392
1.3337	29.0	7250	1.5438	0.9286	0.9309	0.9297	0.9388
1.3329	30.0	7500	1.5388	0.9325	0.9310	0.9318	0.9410
1.3321	31.0	7750	1.5443	0.9319	0.9314	0.9317	0.9408
1.3319	32.0	8000	1.5413	0.9317	0.9334	0.9325	0.9415
1.3313	33.0	8250	1.5428	0.9329	0.9332	0.9331	0.9413
1.3309	34.0	8500	1.5452	0.9288	0.9317	0.9302	0.9396
1.3308	35.0	8750	1.5382	0.9307	0.9324	0.9315	0.9410
1.3307	36.0	9000	1.5370	0.9314	0.9334	0.9324	0.9413
1.33	37.0	9250	1.5391	0.9321	0.9328	0.9325	0.9414
1.3297	38.0	9500	1.5386	0.9330	0.9335	0.9333	0.9414
1.3293	39.0	9750	1.5378	0.9336	0.9343	0.9340	0.9420
1.3294	40.0	10000	1.5378	0.9334	0.9341	0.9337	0.9421

Framework versions

Transformers 4.35.2
Pytorch 2.0.1+cu118
Datasets 2.15.0
Tokenizers 0.15.0

jamesngai/xlm-roberta-xl-final-lora500

xlm-roberta-xl-final-lora500

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

jamesngai/
xlm-roberta-xl-final-lora500