t5-small-codesearchnet-python-stripped – AI Model by lmeninato | AlphaNeural AI

You can deploy this model and start earning money today!

lmeninato/
t5-small-codesearchnet-python-stripped

like

0

text2text-generation

generated_from_trainer

text-generation-inference

endpoints_compatible

Views

No views yet

t5-small-codesearchnet-python

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.7802
Bleu: 0.0027
Rouge1: 0.2097
Rouge2: 0.0628
Avg Length: 15.7496

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 10
total_train_batch_size: 80
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Rouge1	Rouge2	Avg Length
No log	1.0	375	0.9665	0.0004	0.1696	0.0384	14.1472
2.0677	2.0	750	0.8740	0.0004	0.1758	0.0428	13.38
0.8314	3.0	1125	0.8281	0.001	0.1925	0.0505	14.8026
0.7563	4.0	1500	0.7996	0.0017	0.2033	0.0582	14.9606
0.7563	5.0	1875	0.7780	0.0022	0.2117	0.0607	14.9434
0.6959	6.0	2250	0.7587	0.002	0.2135	0.0621	14.6926
0.6591	7.0	2625	0.7545	0.002	0.2073	0.0605	15.2818
0.6205	8.0	3000	0.7472	0.0024	0.2187	0.0674	15.051
0.6205	9.0	3375	0.7506	0.0031	0.2266	0.0696	15.6286
0.5822	10.0	3750	0.7449	0.001	0.206	0.063	13.1462
0.5553	11.0	4125	0.7573	0.0027	0.2148	0.0647	16.4076
0.5295	12.0	4500	0.7677	0.0026	0.2185	0.0658	15.8986
0.5295	13.0	4875	0.7595	0.0009	0.2052	0.0618	12.3528
0.4977	14.0	5250	0.7766	0.0035	0.2177	0.0636	16.6706
0.4818	15.0	5625	0.7802	0.0027	0.2097	0.0628	15.7496

Framework versions

Transformers 4.28.1
Pytorch 2.0.0+cu118
Datasets 2.12.0
Tokenizers 0.13.3