This model is a fine-tuned version of distilbert/distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.4164
F1: 0.6705
Roc Auc: 0.7913
Accuracy: 0.6468

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 7e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 150
num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Roc Auc	Accuracy
No log	1.0	59	0.3846	0.0	0.5	0.4468
No log	2.0	118	0.2943	0.2969	0.5876	0.5191
No log	3.0	177	0.2469	0.5548	0.7060	0.5745
No log	4.0	236	0.2451	0.575	0.7283	0.5787
No log	5.0	295	0.2360	0.6488	0.7739	0.6085
No log	6.0	354	0.2463	0.6190	0.7586	0.5915
No log	7.0	413	0.2724	0.6414	0.7741	0.6213
No log	8.0	472	0.2846	0.6435	0.7764	0.6085
0.1953	9.0	531	0.2961	0.6667	0.7942	0.6426
0.1953	10.0	590	0.3187	0.6627	0.7823	0.6298
0.1953	11.0	649	0.3204	0.6609	0.7874	0.6170
0.1953	12.0	708	0.3497	0.6529	0.7784	0.6298
0.1953	13.0	767	0.3465	0.6589	0.7833	0.6383
0.1953	14.0	826	0.3617	0.6494	0.7813	0.6298
0.1953	15.0	885	0.3759	0.6514	0.7836	0.6383
0.1953	16.0	944	0.3715	0.6512	0.7799	0.6213
0.008	17.0	1003	0.3808	0.6609	0.7856	0.6426
0.008	18.0	1062	0.3850	0.6629	0.7915	0.6383
0.008	19.0	1121	0.3958	0.6553	0.7862	0.6340
0.008	20.0	1180	0.3915	0.6610	0.7893	0.6340
0.008	21.0	1239	0.4016	0.6477	0.7827	0.6255
0.008	22.0	1298	0.4060	0.6496	0.7831	0.6255
0.008	23.0	1357	0.4058	0.6667	0.7923	0.6468
0.008	24.0	1416	0.4119	0.6667	0.7887	0.6468
0.008	25.0	1475	0.4094	0.6648	0.7901	0.6426
0.0021	26.0	1534	0.4151	0.6686	0.7891	0.6511
0.0021	27.0	1593	0.4146	0.6648	0.7901	0.6426
0.0021	28.0	1652	0.4164	0.6705	0.7913	0.6468
0.0021	29.0	1711	0.4174	0.6667	0.7905	0.6426
0.0021	30.0	1770	0.4171	0.6686	0.7928	0.6426

Framework versions

Transformers 4.41.1
Pytorch 2.3.0+cu121
Datasets 2.19.1
Tokenizers 0.19.1

dahe827/distilbert-base-uncased-airlines-news-multi-label

distilbert-base-uncased-airlines-news-multi-label

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

dahe827/
distilbert-base-uncased-airlines-news-multi-label