You can deploy this model and start earning money today!

tuanio/
w2v2_ablation_focal_ctc_a0.99_g2.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16

Name: w2v2_ablation_focal_ctc_a0.99_g2.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16
Availability: InStock

transformers

safetensors

wav2vec2

generated_from_trainer

nguyenvulebinh/wav2vec2-base-vietnamese-250h

finetune

cc-by-nc-4.0

endpoints_compatible

Views

No views yet

w2v2_ablation_focal_ctc_a0.99_g2.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16

This model is a fine-tuned version of nguyenvulebinh/wav2vec2-base-vietnamese-250h on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 3.8640
Wer: 0.0921

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 16
seed: 42
distributed_type: multi-GPU
num_devices: 4
total_train_batch_size: 32
total_eval_batch_size: 64
optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 100
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1766.7398	0.94	100	1156.0825	18.6404
1229.5048	1.89	200	451.0281	17.3500
214.4639	2.83	300	87.0115	1.0
111.4872	3.77	400	79.7089	1.0
102.4801	4.72	500	75.7054	1.0
98.4932	5.66	600	74.1378	1.0
97.156	6.6	700	72.9694	1.0
96.3414	7.55	800	72.1177	1.0
94.0356	8.49	900	71.9697	1.0
92.1795	9.43	1000	68.5141	0.9649
76.3563	10.38	1100	37.3971	0.5475
42.8384	11.32	1200	16.6551	0.2407
27.616	12.26	1300	11.1745	0.1811
20.8896	13.21	1400	9.0025	0.1557
17.9194	14.15	1500	7.6351	0.1422
14.9913	15.09	1600	7.0627	0.1458
13.8141	16.04	1700	6.2267	0.1285
12.5947	16.98	1800	5.8773	0.1296
11.4323	17.92	1900	5.4096	0.1223
11.4649	18.87	2000	5.2264	0.1174
10.2032	19.81	2100	5.2163	0.1262
10.1248	20.75	2200	4.9520	0.1167
9.6892	21.7	2300	4.8752	0.1139
9.2633	22.64	2400	4.7607	0.1123
9.4179	23.58	2500	4.5992	0.1072
8.4707	24.53	2600	4.7147	0.1057
8.54	25.47	2700	4.6794	0.1221
8.0287	26.42	2800	4.6396	0.1285
7.4909	27.36	2900	4.5319	0.1169
7.312	28.3	3000	4.2928	0.1024
7.2176	29.25	3100	4.4070	0.0989
7.3798	30.19	3200	4.3618	0.1097
7.376	31.13	3300	4.2365	0.1014
6.2646	32.08	3400	4.1432	0.0991
6.7574	33.02	3500	4.3109	0.1035
6.6224	33.96	3600	4.0862	0.0954
6.2809	34.91	3700	4.1749	0.0937
6.0949	35.85	3800	4.0581	0.0940
5.873	36.79	3900	4.0394	0.0922
6.379	37.74	4000	4.0111	0.0950
6.2536	38.68	4100	4.0030	0.1030
5.71	39.62	4200	4.0262	0.0987
5.8827	40.57	4300	4.0195	0.0958
5.949	41.51	4400	3.8727	0.0918
6.191	42.45	4500	3.9814	0.0998
5.3531	43.4	4600	3.9668	0.0956
4.7686	44.34	4700	4.0428	0.0932
4.9675	45.28	4800	3.9818	0.0903
4.9916	46.23	4900	3.9620	0.0919
4.9262	47.17	5000	3.8938	0.0902
4.4968	48.11	5100	3.9534	0.0969
4.6931	49.06	5200	3.9595	0.1000
4.3351	50.0	5300	3.9389	0.0947
4.3567	50.94	5400	3.8662	0.0900
4.5551	51.89	5500	3.9161	0.0908
4.711	52.83	5600	3.9064	0.0911
4.6363	53.77	5700	3.9142	0.0939
4.133	54.72	5800	3.8690	0.0922
4.0667	55.66	5900	3.9283	0.0948
4.1538	56.6	6000	3.8803	0.0955
4.2073	57.55	6100	3.8417	0.0928
3.8029	58.49	6200	3.9996	0.1051
4.0944	59.43	6300	3.8510	0.0955
3.834	60.38	6400	3.9362	0.0906
4.2577	61.32	6500	3.9684	0.0948
4.0153	62.26	6600	3.9265	0.0933
4.1588	63.21	6700	3.9087	0.0907
4.0515	64.15	6800	3.8959	0.0973
3.8807	65.09	6900	3.9093	0.0923
3.9373	66.04	7000	3.8751	0.0963
3.7449	66.98	7100	3.8544	0.0923
3.708	67.92	7200	3.8862	0.0913
4.2862	68.87	7300	3.8720	0.0901
3.8561	69.81	7400	3.9259	0.0958
3.4827	70.75	7500	3.8838	0.0948
3.4695	71.7	7600	3.8711	0.0960
3.3134	72.64	7700	3.9087	0.0975
3.3835	73.58	7800	3.8585	0.0953
3.1626	74.53	7900	3.8288	0.0929
3.3835	75.47	8000	3.8568	0.0934
3.5377	76.42	8100	3.8387	0.0933
3.5304	77.36	8200	3.8690	0.0918
3.1716	78.3	8300	3.8811	0.0938
4.0648	79.25	8400	3.8896	0.0965
3.5254	80.19	8500	3.8406	0.0924
3.4569	81.13	8600	3.8227	0.0903
3.5489	82.08	8700	3.8090	0.0897
3.0184	83.02	8800	3.8374	0.0918
3.0423	83.96	8900	3.8569	0.0906
3.1676	84.91	9000	3.8721	0.0921
3.3844	85.85	9100	3.8685	0.0921
3.1564	86.79	9200	3.8815	0.0934
3.3619	87.74	9300	3.8594	0.0924
3.2999	88.68	9400	3.8588	0.0920
3.3061	89.62	9500	3.8653	0.0939
3.3101	90.57	9600	3.8582	0.0923
3.3402	91.51	9700	3.8671	0.0918
3.6063	92.45	9800	3.8736	0.0928
3.2578	93.4	9900	3.8629	0.0926
3.1261	94.34	10000	3.8682	0.0928
3.3517	95.28	10100	3.8634	0.0927
3.292	96.23	10200	3.8680	0.0928
3.5909	97.17	10300	3.8654	0.0925
3.3933	98.11	10400	3.8638	0.0923
2.9496	99.06	10500	3.8647	0.0924
3.4093	100.0	10600	3.8640	0.0921

Framework versions

Transformers 4.35.2
Pytorch 1.13.1+cu117
Datasets 2.12.0
Tokenizers 0.14.1

tuanio/w2v2_ablation_focal_ctc_a0.99_g2.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16

w2v2_ablation_focal_ctc_a0.99_g2.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

tuanio/
w2v2_ablation_focal_ctc_a0.99_g2.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16