This is a DistilBERT model fine-tuned on the AG news dataset to determine whether a news article is within any of the four categories: 'World', 'Sports', 'Business' and 'Sci/Tech'. The Trainer API was used to train the model.

Intended uses & limitations

Training and evaluation data

Data is from HuggingFace's datasets package, source: AG news dataset

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1
0.1888	1.0	3750	0.1786	0.9418	0.9422	0.9418	0.9419
0.1289	2.0	7500	0.1741	0.9459	0.9462	0.9459	0.9460
0.0989	3.0	11250	0.1799	0.9470	0.9472	0.9470	0.9470

Framework versions

Transformers 4.33.0
Pytorch 2.0.0
Datasets 2.1.0
Tokenizers 0.13.3

MuntasirHossain/distilbert-finetuned-ag-news

distilbert-finetuned-ag-news

Model description