6e-5_4000evaltest
This model was trained from scratch on the ami dataset.
It achieves the following results on the evaluation set:
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 6e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- training_steps: 4000
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer |
|---|
| No log | 7.5758 | 250 | 6.5111 | 1.0 |
| 12.7243 | 15.1515 | 500 | 3.3744 | 1.0 |
| 12.7243 | 22.7273 | 750 | 3.0806 | 1.0 |
| 3.3334 | 30.3030 | 1000 | 3.2656 | 1.0 |
| 3.3334 | 37.8788 | 1250 | 2.9244 | 1.0 |
| 2.5277 | 45.4545 | 1500 | 1.2766 | 0.4895 |
| 2.5277 | 53.0303 | 1750 | 1.3037 | 0.3171 |
| 1.0439 | 60.6061 | 2000 | 1.0592 | 0.2950 |
| 1.0439 | 68.1818 | 2250 | 1.0711 | 0.2896 |
| 0.7466 | 75.7576 | 2500 | 1.0562 | 0.2809 |
| 0.7466 | 83.3333 | 2750 | 1.0679 | 0.2822 |
| 0.5899 | 90.9091 | 3000 | 1.1271 | 0.2734 |
| 0.5899 | 98.4848 | 3250 | 1.1434 | 0.2692 |
| 0.4165 | 106.0606 | 3500 | 1.0480 | 0.2683 |
| 0.4165 | 113.6364 | 3750 | 1.1392 | 0.2651 |
| 0.493 | 121.2121 | 4000 | 1.1036 | 0.2644 |
Framework versions
- Transformers 4.42.4
- Pytorch 2.3.1+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1
