loose-default_seed-42_1e-3 – AI Model by qing-yao | AlphaNeural AI

You can deploy this model and start earning money today!

qing-yao/
loose-default_seed-42_1e-3

like

0

text-generation

generated_from_trainer

autotrain_compatible

text-generation-inference

endpoints_compatible

Views

No views yet

loose-default_seed-42_1e-3

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 3.0164
Accuracy: 0.4205

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.001
train_batch_size: 32
eval_batch_size: 64
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 256
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 32000
num_epochs: 20.0
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
6.1722	0.9999	1788	4.2550	0.3059
4.045	1.9999	3576	3.7255	0.3481
3.6123	2.9998	5364	3.4704	0.3713
3.3915	3.9997	7152	3.3354	0.3837
3.305	4.9997	8940	3.2559	0.3910
3.1998	5.9996	10728	3.2065	0.3959
3.1365	6.9995	12516	3.1756	0.3993
3.0938	8.0	14305	3.1521	0.4017
3.0612	8.9999	16093	3.1366	0.4033
3.0164	9.9999	17881	3.1261	0.4045
2.9964	10.9998	19669	3.1167	0.4056
2.987	11.9997	21457	3.1126	0.4064
2.9785	12.9997	23245	3.1036	0.4072
2.9733	13.9996	25033	3.1036	0.4077
2.9329	14.9995	26821	3.0979	0.4085
2.9364	16.0	28610	3.0969	0.4082
2.941	16.9999	30398	3.0937	0.4087
2.9454	17.9999	32186	3.0854	0.4096
2.8897	18.9998	33974	3.0384	0.4160
2.7368	19.9986	35760	3.0164	0.4205

Framework versions

Transformers 4.45.1
Pytorch 2.5.1+cu124
Datasets 2.19.1
Tokenizers 0.20.0