distilroberta-propaganda-2class – AI Model by valurank | AlphaNeural AI

You can deploy this model and start earning money today!

valurank/
distilroberta-propaganda-2class

like

0

text-classification

generated_from_trainer

autotrain_compatible

endpoints_compatible

Views

No views yet

distilroberta-propaganda-2class

This model is a fine-tuned version of distilroberta-base on the QCRI propaganda dataset.

It achieves the following results on the evaluation set:

Loss: 0.5087
Acc: 0.7424

Training and evaluation data

Training data is the 19-class QCRI propaganda data, with all propaganda classes collapsed to a single catch-all 'prop' class.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 32
eval_batch_size: 32
seed: 12345
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 16
num_epochs: 20
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Acc
0.5737	1.0	493	0.5998	0.6515
0.4954	2.0	986	0.5530	0.7080
0.4774	3.0	1479	0.5331	0.7258
0.4846	4.0	1972	0.5247	0.7339
0.4749	5.0	2465	0.5392	0.7199
0.502	6.0	2958	0.5124	0.7466
0.457	7.0	3451	0.5167	0.7432
0.4899	8.0	3944	0.5160	0.7428
0.4833	9.0	4437	0.5280	0.7339
0.5114	10.0	4930	0.5112	0.7436
0.4419	11.0	5423	0.5060	0.7525
0.4743	12.0	5916	0.5031	0.7547
0.4597	13.0	6409	0.5043	0.7517
0.4861	14.0	6902	0.5055	0.7487
0.499	15.0	7395	0.5091	0.7419
0.501	16.0	7888	0.5037	0.7521
0.4659	17.0	8381	0.5087	0.7424

Framework versions

Transformers 4.11.2
Pytorch 1.7.1
Datasets 1.11.0
Tokenizers 0.10.3