This model is a fine-tuned version of google/bert_uncased_L-2_H-128_A-2 on the emotion dataset. It achieves the following results on the evaluation set:

Loss: 0.2502
Accuracy: 0.913
F1: 0.9131

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 64
eval_batch_size: 64
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant
num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.5953	1.0	250	1.4759	0.5055	0.3899
1.3208	2.0	500	1.1113	0.628	0.5554
1.0064	3.0	750	0.8224	0.79	0.7802
0.7535	4.0	1000	0.6185	0.8455	0.8425
0.5891	5.0	1250	0.5004	0.877	0.8758
0.4783	6.0	1500	0.4260	0.8865	0.8862
0.4078	7.0	1750	0.3787	0.8905	0.8903
0.3554	8.0	2000	0.3432	0.891	0.8909
0.3146	9.0	2250	0.3181	0.8925	0.8924
0.2808	10.0	2500	0.2986	0.8965	0.8970
0.2659	11.0	2750	0.2881	0.9	0.8999
0.2487	12.0	3000	0.2740	0.907	0.9072
0.2253	13.0	3250	0.2683	0.9045	0.9047
0.2103	14.0	3500	0.2650	0.9095	0.9099
0.1995	15.0	3750	0.2551	0.9105	0.9108
0.1894	16.0	4000	0.2534	0.9085	0.9088
0.1791	17.0	4250	0.2473	0.91	0.9102
0.168	18.0	4500	0.2441	0.913	0.9134
0.1563	19.0	4750	0.2459	0.9105	0.9107
0.1511	20.0	5000	0.2497	0.9075	0.9076
0.1363	21.0	5250	0.2502	0.913	0.9131

Framework versions

Transformers 4.13.0
Pytorch 1.12.1+cu113
Datasets 1.16.1
Tokenizers 0.10.3

muhtasham/bert_uncased_L-2_H-128_A-2-finetuned-emotion

bert_uncased_L-2_H-128_A-2-finetuned-emotion

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

muhtasham/
bert_uncased_L-2_H-128_A-2-finetuned-emotion