You can deploy this model and start earning money today!

Mardiyyah/
variant-tapt_grouped_llrd-LR_1e-06

Name: variant-tapt_grouped_llrd-LR_1e-06
Availability: InStock

generated_from_trainer

microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext

finetune

mit

endpoints_compatible

Views

No views yet

variant-tapt_grouped_llrd-LR_1e-06

This model is a fine-tuned version of microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext on the Mardiyyah/TAPT-PDBE-V1 dataset. It achieves the following results on the evaluation set:

Loss: 1.6306
Accuracy: 0.6709

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-06
train_batch_size: 16
eval_batch_size: 16
seed: 3407
gradient_accumulation_steps: 4
total_train_batch_size: 64
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-06 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.06
num_epochs: 100
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
2.0274	1.0	116	1.9458	0.6314
1.9887	2.0	232	1.8570	0.6395
1.9199	3.0	348	1.8270	0.6442
1.8141	4.0	464	1.7926	0.6430
1.7675	5.0	580	1.7232	0.6516
1.7878	6.0	696	1.7268	0.6545
1.7164	7.0	812	1.6792	0.6619
1.7193	8.0	928	1.7039	0.6526
1.6903	9.0	1044	1.7040	0.6522
1.6839	10.0	1160	1.6893	0.6597
1.6813	11.0	1276	1.6971	0.6555
1.6952	12.0	1392	1.6419	0.6621
1.6786	13.0	1508	1.6864	0.6613
1.6776	14.0	1624	1.6605	0.6636
1.6491	15.0	1740	1.7014	0.6621
1.6355	16.0	1856	1.6744	0.6625
1.6444	17.0	1972	1.6295	0.6681
1.647	18.0	2088	1.6468	0.6625
1.6424	19.0	2204	1.6326	0.6685
1.6413	20.0	2320	1.6026	0.6661
1.6373	21.0	2436	1.6191	0.6636
1.6346	22.0	2552	1.6341	0.6680
1.6292	23.0	2668	1.6228	0.6637
1.6228	24.0	2784	1.6334	0.6676
1.6097	25.0	2900	1.6086	0.6673
1.6294	26.0	3016	1.6440	0.6667
1.6092	27.0	3132	1.6509	0.6659
1.6283	28.0	3248	1.6326	0.6642
1.6079	29.0	3364	1.6422	0.6700
1.6089	30.0	3480	1.6440	0.6631
1.6113	31.0	3596	1.6147	0.6652
1.6092	32.0	3712	1.5883	0.6710
1.6241	33.0	3828	1.5963	0.6732
1.6115	34.0	3944	1.6352	0.6697
1.5853	35.0	4060	1.6446	0.6667
1.5991	36.0	4176	1.6407	0.6620
1.5932	37.0	4292	1.6916	0.6571
1.5637	38.0	4408	1.6387	0.6661
1.5815	39.0	4524	1.6136	0.6719
1.567	40.0	4640	1.6329	0.6662
1.5819	41.0	4756	1.5787	0.6738
1.5715	42.0	4872	1.6160	0.6688
1.5965	43.0	4988	1.5861	0.6716
1.6106	44.0	5104	1.5932	0.6731
1.5789	45.0	5220	1.6258	0.6652
1.5746	46.0	5336	1.6581	0.6650
1.5534	47.0	5452	1.5644	0.6766
1.5937	48.0	5568	1.6418	0.6628
1.5737	49.0	5684	1.6214	0.6746
1.5796	50.0	5800	1.6438	0.6676
1.5635	51.0	5916	1.6004	0.6709
1.5889	52.0	6032	1.6668	0.6667
1.5334	53.0	6148	1.6160	0.6678
1.5839	54.0	6264	1.6121	0.6684
1.5572	55.0	6380	1.5689	0.6776
1.596	56.0	6496	1.6458	0.6591
1.5745	57.0	6612	1.5974	0.6690
1.5824	58.0	6728	1.7037	0.6552
1.5816	59.0	6844	1.5671	0.6768
1.5797	60.0	6960	1.5961	0.6775
1.5757	61.0	7076	1.5632	0.6750
1.5734	62.0	7192	1.6264	0.6706
1.5396	63.0	7308	1.6148	0.6728
1.5696	64.0	7424	1.6255	0.6684
1.5705	65.0	7540	1.6008	0.6717
1.5703	66.0	7656	1.6372	0.6636
1.5496	67.0	7772	1.6201	0.6658
1.5642	68.0	7888	1.6311	0.6669
1.5697	69.0	8004	1.6085	0.6667
1.5645	70.0	8120	1.6461	0.6645
1.5563	71.0	8236	1.6093	0.6721
1.5636	72.0	8352	1.6316	0.6647
1.5413	73.0	8468	1.6773	0.6629
1.5517	74.0	8584	1.6432	0.6598
1.5695	75.0	8700	1.6504	0.6641
1.5692	76.0	8816	1.6190	0.6694
1.5608	77.0	8932	1.6546	0.6632
1.5735	78.0	9048	1.5984	0.6699
1.5319	79.0	9164	1.6111	0.6710
1.5683	80.0	9280	1.6204	0.6665
1.5756	81.0	9396	1.6569	0.6608
1.5659	82.0	9512	1.6072	0.6746
1.5614	83.0	9628	1.6069	0.6741
1.5598	84.0	9744	1.6796	0.6601
1.5615	85.0	9860	1.6378	0.6645
1.561	86.0	9976	1.6115	0.6738
1.5644	87.0	10092	1.6623	0.6609
1.5678	88.0	10208	1.6449	0.6616
1.5674	89.0	10324	1.6009	0.6719
1.5656	90.0	10440	1.6009	0.6710
1.5458	91.0	10556	1.5955	0.6758
1.5759	92.0	10672	1.6359	0.6671
1.5829	93.0	10788	1.6427	0.6672
1.5687	94.0	10904	1.5788	0.6706
1.5697	95.0	11020	1.5978	0.6704
1.5614	96.0	11136	1.6515	0.6670
1.5706	97.0	11252	1.6041	0.6708
1.5612	98.0	11368	1.6063	0.6691
1.5795	99.0	11484	1.6268	0.6652
1.6168	99.1382	11500	1.6341	0.6659

Framework versions

Transformers 4.48.2
Pytorch 2.4.1+cu121
Datasets 3.0.2
Tokenizers 0.21.0

Mardiyyah/variant-tapt_grouped_llrd-LR_1e-06

variant-tapt_grouped_llrd-LR_1e-06

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Mardiyyah/
variant-tapt_grouped_llrd-LR_1e-06