Whisper Large-v3 Tunisian Arabic LoRA
montachouig/whisper-large-v3-tunisian is a LoRA adapter fine-tuned from
openai/whisper-large-v3 for automatic speech
recognition (ASR) in Tunisian Arabic. It is an adapter, so the base model is
downloaded when the model is loaded.
Model details
- Architecture: Whisper Large-v3 encoder-decoder Transformer
- Fine-tuning method: PEFT LoRA
- Language: Arabic / Tunisian Arabic (
ar)
- Task: transcription
- Base model:
openai/whisper-large-v3
- Adapter rank: 32
- LoRA alpha: 32
- LoRA dropout: 0.05
- Target modules:
out_proj, fc2, q_proj, v_proj, k_proj, fc1
- Bias:
none
Data preperation and augmentation
The augmentation pipeline improves speech-recognition robustness by creating additional
audio samples with synthetic room reverberation and white, pink, or brown background noise at randomized SNR levels.
It preserves the original transcripts, segmentation, duration,
and schema, while re-encoding augmented audio as WAV and retaining the original samples for fine-tuning.
Training
The training data consists of audio and Tunisian Arabic transcript pairs. A
reproducible 10% holdout split
was created with seed 42. Dataset identity,
license, size, and collection methodology were not recorded in the checkpoint;
users should obtain those details from the model author before deployment.
| Hyperparameter | Value |
|---|
| Checkpoint step | 4,500 |
| Epochs configured | 10 |
| Learning rate | 0.0001 |
| Train batch size/device | 18 |
| Gradient accumulation | 4 |
| Effective batch size/device | 72 |
| Warmup steps | 500 |
| Weight decay | 0.0 |
| Precision | BF16 |
| Generation beams | 4 |
| Maximum generated tokens | 225 |
The best checkpoint reported by Trainer is outputs/whisper-large-v3-tunisian-arabic-lora/checkpoint-4500. The selected
validation metric is WER (lower is better); its recorded value is 21.743907.
WER/CER figures should not be interpreted without the held-out dataset details.
Usage
1from peft import PeftModel
2from transformers import WhisperForConditionalGeneration
3
4base = WhisperForConditionalGeneration.from_pretrained(
5 "openai/whisper-large-v3",
6 torch_dtype="float32",
7 device_map=None, # avoid bnb/accelerate quantized loading here
8)
9
10merged = PeftModel.from_pretrained(base, "montachouig/whisper-large-v3-tunisian").merge_and_unload()
11merged.save_pretrained("/path/to/merged-finetuned-whisper-v3")
12
Install the runtime dependencies with pip install transformers peft accelerate torch soundfile.
Intended use and limitations
This model is intended for research and transcription of Tunisian Arabic speech.
Accuracy may degrade for other Arabic varieties, code-switching, noisy audio,
unseen accents, specialized vocabulary, overlapping speakers, or long recordings.
Whisper can hallucinate plausible text, so output should be reviewed by a human
for medical, legal, safety-critical, or other high-impact uses. Do not use the
model to infer sensitive traits or to transcribe people without an appropriate
legal basis and consent.
Evaluation and reproducibility
The repository contains adapter and processor artifacts needed for inference.
Data source
The LINAGORA/LinTO Dataset Audio AR-TN was used to fine-tune the model.
