micro_base_help_tapt_pretrain_model – AI Model by BigTMiami | AlphaNeural AI

You can deploy this model and start earning money today!

BigTMiami/
micro_base_help_tapt_pretrain_model

like

0

generated_from_trainer

FacebookAI/roberta-base

autotrain_compatible

endpoints_compatible

Views

No views yet

micro_base_help_tapt_pretrain_model

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.5916

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 21
eval_batch_size: 21
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 42
optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
lr_scheduler_type: linear
num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss
1.9109	0.99	40	1.6849
1.7421	2.0	81	1.6620
1.7411	2.99	121	1.6333
1.6441	4.0	162	1.6306
1.6337	4.99	202	1.6137
1.5774	6.0	243	1.6343
1.5997	6.99	283	1.5931
1.5196	8.0	324	1.6018
1.5416	8.99	364	1.5994
1.4819	10.0	405	1.5886
1.5079	10.99	445	1.5938
1.455	12.0	486	1.5699
1.4718	12.99	526	1.5947
1.4157	14.0	567	1.5920
1.4369	14.99	607	1.5879
1.3733	16.0	648	1.5745
1.4017	16.99	688	1.6000
1.3601	18.0	729	1.5830
1.3602	18.99	769	1.5846
1.3152	20.0	810	1.5940
1.3437	20.99	850	1.5942
1.2904	22.0	891	1.5787

Framework versions

Transformers 4.36.2
Pytorch 2.2.1+cu121
Datasets 2.19.0
Tokenizers 0.15.2