SCIFACT_xlm_roberta_large – AI Model by nikolamilosevic | AlphaNeural AI

You can deploy this model and start earning money today!

nikolamilosevic/
SCIFACT_xlm_roberta_large

like

0

text-classification

generated_from_trainer

allenai/scifact

FacebookAI/xlm-roberta-large

autotrain_compatible

endpoints_compatible

Views

No views yet

SCIFACT_inference_model

This model is a fine-tuned version of xlm-roberta-large on the SciFact dataset. It achieves the following results on the evaluation set:

Loss: 1.2496
Accuracy: 0.8819

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 3
eval_batch_size: 3
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	378	1.0485	0.4724
1.0382	2.0	756	1.3964	0.6063
0.835	3.0	1134	0.9168	0.8268
0.6801	4.0	1512	0.7524	0.8425
0.6801	5.0	1890	1.0672	0.8346
0.4291	6.0	2268	0.9599	0.8425
0.2604	7.0	2646	0.8691	0.8661
0.1932	8.0	3024	1.3162	0.8268
0.1932	9.0	3402	1.3200	0.8583
0.0974	10.0	3780	1.1566	0.8740
0.1051	11.0	4158	1.1568	0.8819
0.0433	12.0	4536	1.2013	0.8661
0.0433	13.0	4914	1.1557	0.8819
0.034	14.0	5292	1.3044	0.8661
0.0303	15.0	5670	1.2496	0.8819

Framework versions

Transformers 4.34.1
Pytorch 1.13.1+cu116
Datasets 2.14.6
Tokenizers 0.14.1