You can deploy this model and start earning money today!

Tsedeniya/
finetuned_byT5base_jje

Name: finetuned_byT5base_jje
Availability: InStock

transformers

safetensors

text2text-generation

generated_from_trainer

google/byt5-base

finetune

apache-2.0

text-generation-inference

endpoints_compatible

Views

No views yet

finetuned_byT5base_jje

This model is a fine-tuned version of google/byt5-base on a Jejuo dataset trained on native script. It achieves the following results on the evaluation set:

Loss: 0.9066
Chrf++: 61.5944
Gen Len: 89.3128

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 32
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Chrf++	Gen Len
31.9871	2.0	44	3.4258	4.3897	188.4645
31.9871	4.0	88	2.9267	3.6415	188.763
29.1995	6.0	132	2.1996	5.5132	188.8815
29.1995	8.0	176	1.4532	9.7154	184.7536
15.0082	10.0	220	1.1236	15.4914	149.8152
15.0082	12.0	264	1.0400	17.777	142.4834
9.3270	14.0	308	0.9843	12.1173	444.3223
9.3270	16.0	352	0.8906	17.5524	166.4076
9.3270	18.0	396	0.8660	18.0981	197.7299
7.1078	20.0	440	0.8365	16.2761	319.0142
7.1078	22.0	484	0.7756	23.0987	220.2464
5.5376	24.0	528	0.7554	23.8728	251.9289
5.5376	26.0	572	0.7087	27.2003	253.327
4.1454	28.0	616	0.6730	37.6135	155.6209
4.1454	30.0	660	0.6675	34.7199	276.6777
2.8138	32.0	704	0.6716	44.9468	161.3223
2.8138	34.0	748	0.6600	47.8239	125.6114
2.8138	36.0	792	0.6951	52.6598	111.0664
1.8489	38.0	836	0.7095	54.4166	106.6635
1.8489	40.0	880	0.7138	56.9882	107.8057
1.2351	42.0	924	0.7128	56.2904	87.3033
1.2351	44.0	968	0.7687	59.5219	95.9431
0.8148	46.0	1012	0.7835	59.8042	108.5545
0.8148	48.0	1056	0.8140	59.9867	88.9763
0.5622	50.0	1100	0.8142	61.3704	92.8957
0.5622	52.0	1144	0.8402	62.0918	93.9005
0.5622	54.0	1188	0.8712	60.391	100.7109
0.4025	56.0	1232	0.8419	61.0309	96.545
0.4025	58.0	1276	0.8804	62.4253	98.9953
0.3107	60.0	1320	0.9094	61.5251	93.1137
0.3107	62.0	1364	0.9180	61.3171	89.7725
0.2653	64.0	1408	0.9066	61.5944	89.3128

Framework versions

Transformers 5.12.1
Pytorch 2.6.0+cu124
Datasets 5.0.0
Tokenizers 0.22.2

Tsedeniya/finetuned_byT5base_jje

finetuned_byT5base_jje

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Tsedeniya/
finetuned_byT5base_jje