albert-xxlarge-v2-Adapters – AI Model by dhanishetty | AlphaNeural AI

You can deploy this model and start earning money today!

dhanishetty/
albert-xxlarge-v2-Adapters

like

0

generated_from_trainer

albert/albert-xxlarge-v2

Views

No views yet

Visualize in Weights & Biases

albert-xxlarge-v2-Adapters

This model is a fine-tuned version of albert/albert-xxlarge-v2 on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.5628

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 64
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: inverse_sqrt
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.121	0.1146	50	1.0992
1.1183	0.2292	100	1.0918
1.1155	0.3438	150	1.0899
1.0405	0.4585	200	0.9750
0.9	0.5731	250	0.9008
0.8362	0.6877	300	0.8511
0.7714	0.8023	350	0.8039
0.7385	0.9169	400	0.7617
0.7422	1.0315	450	0.7265
0.6513	1.1461	500	0.7158
0.7349	1.2607	550	0.6831
0.6515	1.3754	600	0.6679
0.6054	1.4900	650	0.6465
0.6069	1.6046	700	0.6364
0.6132	1.7192	750	0.6344
0.6195	1.8338	800	0.6366
0.6026	1.9484	850	0.6313
0.507	2.0630	900	0.5977
0.5555	2.1777	950	0.5871
0.5855	2.2923	1000	0.5835
0.5642	2.4069	1050	0.5956
0.5937	2.5215	1100	0.5791
0.5763	2.6361	1150	0.5756
0.5041	2.7507	1200	0.5729
0.5753	2.8653	1250	0.5720
0.5546	2.9799	1300	0.5628

Framework versions

PEFT 0.11.1
Transformers 4.42.4
Pytorch 2.1.0
Datasets 2.20.0
Tokenizers 0.19.1