fine-tuned-DatasetQAS-Squad-ID-with-indobert-large-p2-with-ITTL-without-freeze-LR-1e-05 – AI Model by afaji | AlphaNeural AI

You can deploy this model and start earning money today!

afaji/
fine-tuned-DatasetQAS-Squad-ID-with-indobert-large-p2-with-ITTL-without-freeze-LR-1e-05

like

0

question-answering

generated_from_trainer

endpoints_compatible

Views

No views yet

fine-tuned-DatasetQAS-Squad-ID-with-indobert-large-p2-with-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indobenchmark/indobert-large-p2 on the None dataset. It achieves the following results on the evaluation set:

Loss: 1.7872
Exact Match: 46.1526
F1: 62.7803
Precision: 64.0856
Recall: 68.4802

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 64
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 16

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1	Precision	Recall
1.7454	0.5	1024	1.8066	39.3967	56.5617	57.9148	64.1074
1.585	1.0	2048	1.6658	42.3134	59.7125	61.0679	67.5221
1.391	1.5	3072	1.6455	43.6763	60.8729	62.4910	67.0820
1.3937	2.0	4096	1.6125	44.9892	62.0278	63.7779	67.7429
1.1464	2.5	5120	1.6549	44.9892	61.6232	62.7582	68.2833
1.1611	3.0	6144	1.6369	45.5958	62.8723	64.0481	68.8869
1.0415	3.5	7168	1.7452	45.7038	62.3827	63.6067	68.6888
0.9972	4.0	8192	1.7086	45.7288	62.5511	64.1824	67.9289
0.9043	4.5	9216	1.7872	46.1526	62.7803	64.0856	68.4802

Framework versions

Transformers 4.27.0
Pytorch 2.0.0+cu117
Datasets 2.2.0
Tokenizers 0.13.2