anliR2-gpt2-kd – AI Model by asun17904 | AlphaNeural AI

You can deploy this model and start earning money today!

asun17904/
anliR2-gpt2-kd

like

0

Views

No views yet

Knowledge Continuity Regularized Network

Dataset: ANLI Round: None

Trainer Hyperparameters:

lr = 5e-05
per_device_batch_size = 8
gradient_accumulation_steps = 1
weight_decay = 0.0
seed = 42

Regularization Hyperparameters

numerical stability denominator constant = 0.01
lambda = 0.001
alpha = 2.0
beta = 2.0

Extended Logs:

eval_loss	eval_accuracy	epoch
36.424	0.388	1.0
36.257	0.398	2.0
36.011	0.413	3.0
36.333	0.394	4.0
36.104	0.406	5.0
35.441	0.432	6.0
35.057	0.443	7.0
35.235	0.434	8.0
34.987	0.448	9.0
34.987	0.447	10.0
34.540	0.464	11.0
34.358	0.469	12.0
34.304	0.470	13.0
34.409	0.466	14.0
34.153	0.475	15.0
34.440	0.465	16.0
34.267	0.473	17.0
34.298	0.470	18.0
33.984	0.480	19.0

Test Accuracy: 0.481