Model description

More information needed

Intended uses & limitations

More information needed

More information needed

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 1
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 4
total_train_batch_size: 16
total_eval_batch_size: 4
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 2

Training Loss	Epoch	Step	Validation Loss
0.5744	0.0900	200	0.5791
0.5996	0.1800	400	0.5954
0.5826	0.2699	600	0.5893
0.5664	0.3599	800	0.5740
0.5879	0.4499	1000	0.5603
0.5524	0.5399	1200	0.5529
0.555	0.6299	1400	0.5391
0.5104	0.7198	1600	0.5253
0.5478	0.8098	1800	0.5158
0.4864	0.8998	2000	0.5046
0.5385	0.9898	2200	0.4930
0.3133	1.0796	2400	0.5087
0.3397	1.1696	2600	0.5036
0.3044	1.2596	2800	0.4970
0.2874	1.3496	3000	0.4932
0.2925	1.4395	3200	0.4873
0.3034	1.5295	3400	0.4794
0.3035	1.6195	3600	0.4744
0.2727	1.7095	3800	0.4733
0.2729	1.7995	4000	0.4723
0.2715	1.8894	4200	0.4709
0.2846	1.9794	4400	0.4707