dgx1_w2v2_base_teacher_student_distillation_mozilla_epochs_100_batch_16 – AI Model by rohitp1 | AlphaNeural AI

You can deploy this model and start earning money today!

rohitp1/
dgx1_w2v2_base_teacher_student_distillation_mozilla_epochs_100_batch_16

like

0

automatic-speech-recognition

generated_from_trainer

mozilla-foundation/common_voice_11_0

endpoints_compatible

Views

No views yet

dgx1_w2v2_base_teacher_student_distillation_mozilla_epochs_100_batch_16

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

Loss: 33.1259
Wer: 0.9996

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 16
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 128
total_train_batch_size: 2048
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.2
num_epochs: 100
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
81.3268	7.35	500	27.8835	0.9996
62.2215	14.7	1000	28.9444	0.9997
58.4422	22.06	1500	30.4689	0.9997
55.5853	29.41	2000	30.3907	0.9996
53.3674	36.76	2500	31.5643	0.9996
51.3963	44.12	3000	31.6585	0.9996
49.5487	51.47	3500	33.0177	0.9996
48.3909	58.82	4000	33.6293	0.9996
47.6249	66.18	4500	33.1259	0.9996

Framework versions

Transformers 4.25.1
Pytorch 1.12.1
Datasets 2.8.0
Tokenizers 0.13.2