Sagarmatha V4 is a fine-tuned version of openai/whisper-large-v3-turbo for Nepali (Devanagari script) automatic speech recognition. It was trained on 265 hours of curated Nepali audio using parameter-efficient fine-tuning (QLoRA) and achieves a Word Error Rate of 30.99% on a held-out test set, representing approximately a 20 percentage point improvement over the zero-shot baseline.

Model Details

Model Description

Attribute	Value
Model type	Encoder-decoder (Whisper architecture)
Base model	openai/whisper-large-v3-turbo
Fine-tuning method	QLoRA (LoRA rank 32, alpha 64, NF4 quantization)
Language	Nepali (ne)
License	Apache 2.0
Release date	July 2026

This repository contains the fully merged model: the LoRA adapter weights have been folded into the base model parameters. No PEFT library is required at inference time.

Intended Use

Transcription of spoken Nepali audio to Devanagari text
Research on low-resource South Asian language speech recognition
Downstream NLP pipelines requiring Nepali ASR

Out-of-Scope Use

Translation tasks (the model was fine-tuned for transcription only)
Languages other than Nepali
Real-time streaming transcription without chunking for audio longer than 30 seconds

Evaluation Results

Evaluated on held-out splits of the Sagarmatha V4 dataset after 10,000 training steps (~3.5 epochs). Two splits were used: a random sample across all clip lengths, and a long-clip split restricted to utterances of more than 25 words.

Evaluation Split	WER	CER
Random sample (all lengths)	30.99%	9.58%
Long clips (>25 words)	31.70%	10.20%
Baseline: zero-shot whisper-large-v3-turbo	~52%	-

The near-identical WER between short and long clips confirms that the EOS truncation bias present in earlier model versions (Sagarmatha V3, WER 55.74% with 44% deletions on long clips) has been fully resolved in this version.

Usage

Direct Inference

python

from transformers import pipeline

pipe = pipeline(
    "automatic-speech-recognition",
    model="tonibirat/sagarmatha-v4-nepali-asr",
    generate_kwargs={"language": "nepali", "task": "transcribe"},
    device=0,  # use -1 for CPU
)

result = pipe("audio.wav")
print(result["text"])

Long-Form Audio (recommended for audio longer than 30 seconds)

python

result = pipe(
    "long_audio.wav",
    return_timestamps=True,
    chunk_length_s=30,
    stride_length_s=5,
)
print(result["text"])

for chunk in result["chunks"]:
    start, end = chunk["timestamp"]
    print(f"[{start:.1f}s - {end:.1f}s]  {chunk['text']}")

With WhisperProcessor Directly

python

import torch
from transformers import AutoProcessor, WhisperForConditionalGeneration

processor = AutoProcessor.from_pretrained("tonibirat/sagarmatha-v4-nepali-asr")
model = WhisperForConditionalGeneration.from_pretrained(
    "tonibirat/sagarmatha-v4-nepali-asr",
    torch_dtype=torch.float16,
    device_map="auto",
)

# Prepare audio (16 kHz mono expected)
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")
predicted_ids = model.generate(
    inputs["input_features"].to(model.device, dtype=torch.float16),
    language="nepali",
    task="transcribe",
)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
print(transcription[0])

Training Details

Dataset

The model was trained on Sagarmatha V4, a curated corpus of 265 hours of Nepali speech comprising 156,375 audio clips in FLAC format.

Attribute	Value
Total duration	265 hours
Number of clips	156,375
Audio format	FLAC, 16 kHz mono
Sources	OpenSLR-54 (Nepali speech), Internal curated v3
Quality filter	CTC confidence score >= 0.70
Evaluation split	Random held-out set + long-clip set (>25 words)

Data from the OpenSLR source was obtained from openslr.org/54 (Nepali TTS and ASR corpus).

Hyperparameters

Parameter	Value
Base model	openai/whisper-large-v3-turbo
Fine-tuning method	QLoRA
LoRA rank	32
LoRA alpha	64
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, out_proj, fc1, fc2
Quantization	NF4 (4-bit)
Training steps	10,000
Effective epochs	~3.5
Effective batch size	32 (per-device batch 2, gradient accumulation 16)
Learning rate	1e-5
LR schedule	Linear decay with 1,000 warmup steps
Optimizer	AdamW
Final training loss	0.5849

Hardware

Attribute	Value
Hardware	2x NVIDIA Tesla T4 (Kaggle, 16 GB VRAM each)
Training duration	~10.3 hours
Framework	Hugging Face Transformers 4.x + PEFT
Precision	Mixed precision (fp16)

Limitations

Accent coverage: The model was trained predominantly on standard spoken Nepali. Performance on strong regional accents or dialects has not been evaluated.
Code-switching: Utterances mixing Nepali and English may produce degraded output, as the training corpus does not contain code-switched speech.
Noisy environments: No noise augmentation was applied during training. Performance in high-noise conditions is expected to be lower.
Long-form audio: Clips longer than 30 seconds require chunked inference. End-to-end transcription of arbitrarily long audio without chunking is not supported.
Translation: The model was not fine-tuned for translation. Invoking task="translate" will produce Whisper's default translation behaviour, not a fine-tuned translation system.

Citation

If you use this model in your research, please cite the following:

bibtex

@misc{birat2026sagarmatha,
  author       = {Birat, Toni},
  title        = {Sagarmatha V4: QLoRA Fine-Tuning of Whisper for Low-Resource Nepali ASR},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/tonibirat/sagarmatha-v4-nepali-asr}},
}

Acknowledgements

Base model weights are from openai/whisper-large-v3-turbo. Training was conducted on Kaggle (free GPU tier, T4 hardware) using the Hugging Face PEFT and Transformers libraries.