anliR1-gpt2-kd – AI Model by asun17904 | AlphaNeural AI

You can deploy this model and start earning money today!

asun17904/
anliR1-gpt2-kd

like

0

Views

No views yet

Knowledge Continuity Regularized Network

Dataset: ANLI Round: None

Trainer Hyperparameters:

lr = 5e-05
per_device_batch_size = 8
gradient_accumulation_steps = 1
weight_decay = 0.0
seed = 42

Regularization Hyperparameters

numerical stability denominator constant = 0.01
lambda = 0.001
alpha = 2.0
beta = 2.0

Extended Logs:

eval_loss	eval_accuracy	epoch
37.124	0.356	1.0
35.453	0.414	2.0
36.455	0.393	3.0
35.682	0.417	4.0
36.248	0.397	5.0
35.343	0.431	6.0
35.734	0.413	7.0
35.896	0.414	8.0
35.431	0.434	9.0
35.405	0.425	10.0
34.976	0.445	11.0
35.522	0.434	12.0
35.389	0.436	13.0
35.805	0.422	14.0
35.435	0.434	15.0
35.751	0.423	16.0
35.567	0.430	17.0
35.204	0.439	18.0
35.490	0.430	19.0

Test Accuracy: 0.430