Model description

More information needed

Intended uses & limitations

Not intended for real-world use, was an example of MLM fine-tuning on a small radiology dataset.

More information needed

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 20

Training Loss	Epoch	Step	Validation Loss
1.8693	1.0	248	1.5996
1.6968	2.0	496	1.7973
1.7187	3.0	744	1.7232
1.6518	4.0	992	1.7343
1.5003	5.0	1240	1.7727
1.3346	6.0	1488	1.7357
1.4029	7.0	1736	1.7164
1.2762	8.0	1984	1.7123
1.2441	9.0	2232	1.6978
1.2016	10.0	2480	1.7374
1.1887	11.0	2728	1.7076
1.0205	12.0	2976	1.6736
1.0771	13.0	3224	1.7209
1.0607	14.0	3472	1.6753
0.909	15.0	3720	1.6172
0.9255	16.0	3968	1.7418
0.8676	17.0	4216	1.6914
0.8533	18.0	4464	1.7310
0.845	19.0	4712	1.7893
0.869	20.0	4960	1.6936