You can deploy this model and start earning money today!

birajsubedi/
whisper-large-v2-nepali-financial

Name: whisper-large-v2-nepali-financial
Availability: InStock

peft

safetensors

1910.09700

openai/whisper-large-v2

adapter

Views

No views yet

Model Card for Whisper Large-v2 + LoRA (Nepali Financial)

This is a domain-adapted version of Whisper large-v2 fine-tuned with LoRA on the NepFinSpeech-403 dataset for low-resource Nepali financial speech recognition. It is designed to recognize spoken commands for mobile wallet operations (send, load, balance) and handles Devanagari numerals.

The model significantly reduces word error rate (WER) on financial utterances compared to the zero-shot baseline, and is deployed in the SpeakPay voice-first e-wallet application.

Performance

Evaluation on the held-out test set of NepFinSpeech-403:

Model	WER% (↓)	CER% (↓)	NumAcc% (↑)
Whisper large-v2 (zero-shot)	129.95	92.32	0.0
Whisper small (general Nepali FT)	106.32	63.48	0.0
Whisper large-v2 + LoRA (ours)	42.58	16.95	73.9

Relative WER reduction of 67.2% over the zero-shot baseline, with improvements on 59/60 test utterances (sign test, p = 3.5×10⁻¹⁸). Transaction Success Rate improves from 1.67% to 33.33%.

Usage

This is a PEFT adapter and must be loaded together with the base model openai/whisper-large-v2. For usage examples and full training/inference pipeline, please refer to the SpeakPay GitHub repository.

Model Details

Base model: openai/whisper-large-v2
Fine-tuning method: LoRA (rank 32, alpha 64)
Task: Automatic speech recognition (Nepali financial domain)
Language: Nepali
License: MIT (code), CC-BY 4.0 (dataset)

Model Sources

Repository: github.com/subedibiraj/speakpay
Paper: SpeakPay: Domain-Adaptive LoRA Fine-Tuning of Whisper for Low-Resource Nepali Financial Speech Recognition
Demo: speakpay.biraj-subedi.com.np
Dataset: huggingface.co/datasets/birajsubedi/NepFinSpeech

Evaluation

The model was evaluated on the held‑out test set of NepFinSpeech, containing 60 utterances spanning all command types and numeral ranges. Metrics reported are word error rate (WER), character error rate (CER), and Devanagari numeral accuracy (NumAcc). Full analysis (including confusion patterns and data‑efficiency sweeps) is available in the paper and in the analysis/ directory of the repository.

Training Procedure

The LoRA adapter was fine-tuned using the training/ pipeline from the SpeakPay repository. Hyperparameters and details can be found in the repository documentation.

Framework versions

PEFT 0.6.2

Citation

bibtex

@misc{subedi2026speakpay,
  title   = {SpeakPay: Domain-Adaptive LoRA Fine-Tuning of Whisper for
             Low-Resource Nepali Financial Speech Recognition},
  author  = {Subedi, Biraj},
  year    = {2026},
  url     = {https://github.com/subedibiraj/speakpay}
}

birajsubedi/whisper-large-v2-nepali-financial