from transformers import pipeline

transcriber = pipeline(
  "automatic-speech-recognition", 
  model="kingabzpro/whisper-tiny-urdu"
)

transcriber.model.generation_config.forced_decoder_ids = None
transcriber.model.generation_config.language = "ur"

transcription = transcriber("audio2.mp3")
print(transcription)

{'text': 'دیکھیے پانی کب تک بہتا اور مچھلی کب تک تیرتی ہے'}

Evaluation

Dataset	WER (%)	CER (%)	BLEU	ChrF
Common Voice 17.0 (Urdu)	46.908	18.543	32.631	63.988
HowMannyMore/urdu-audiodataset	51.405	21.830	31.475	64.204

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 200
training_steps: 2500
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.6808	1.6949	500	0.7403	52.6699
0.3948	3.3898	1000	0.6850	47.1247
0.2873	5.0847	1500	0.6994	48.1516
0.2024	6.7797	2000	0.7169	46.7326
0.183	8.4746	2500	0.7225	47.8529

Framework versions

Transformers 4.51.3
Pytorch 2.6.0+cu124
Datasets 3.6.0
Tokenizers 0.21.1