This model is a fine-tuned version of facebook/w2v-bert-2.0 on the common_voice_11_0 dataset. It achieves the following results on the evaluation set:

Loss: inf
Wer: 0.1749

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 4
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 32
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 500
num_epochs: 10
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.2117	0.3820	300	inf	0.4642
0.3565	0.7640	600	inf	0.3370
0.2693	1.1461	900	inf	0.2812
0.2215	1.5281	1200	inf	0.2633
0.2138	1.9101	1500	inf	0.2523
0.1676	2.2922	1800	inf	0.2448
0.1569	2.6742	2100	inf	0.2280
0.1516	3.0563	2400	inf	0.2252
0.1192	3.4383	2700	inf	0.2212
0.1198	3.8203	3000	inf	0.2104
0.1051	4.2025	3300	inf	0.2057
0.0925	4.5844	3600	inf	0.2056
0.0916	4.9664	3900	inf	0.2052
0.0763	5.3486	4200	inf	0.1936
0.0722	5.7305	4500	inf	0.1872
0.0675	6.1127	4800	inf	0.1875
0.0554	6.4947	5100	inf	0.1871
0.0577	6.8767	5400	inf	0.1810
0.0456	7.2588	5700	inf	0.1862
0.0418	7.6408	6000	inf	0.1789
0.0401	8.0229	6300	inf	0.1754
0.0293	8.4049	6600	inf	0.1768
0.029	8.7869	6900	inf	0.1766
0.0265	9.1690	7200	inf	0.1753
0.0203	9.5510	7500	inf	0.1750
0.019	9.9330	7800	inf	0.1749

Framework versions

Transformers 4.46.2
Pytorch 2.5.1
Datasets 3.1.0
Tokenizers 0.20.3

mtsotras/w2v-bert-2.0-bengali-colab-CV16.0

w2v-bert-2.0-bengali-colab-CV16.0

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

mtsotras/
w2v-bert-2.0-bengali-colab-CV16.0