Whisper Darija Flights Stage 2
This model is a fine-tuned version of ilyaslbern7347/whisper-darija-stage1 using a custom dataset of Moroccan Darija flight-related speech.
The objective of this model is to improve speech recognition accuracy for Moroccan Darija users searching for flights using natural language.
Model Description
This model is based on OpenAI Whisper and has been further fine-tuned specifically for Moroccan Darija speech in the travel domain.
The model recognizes spoken requests such as:
- Book a flight
- Search for flights
- Departure city
- Destination city
- Travel date
- Number of passengers
- Flight-related conversational requests
This model is intended to be integrated into intelligent flight booking assistants.
Base Model
- Base Model: ilyaslbern7347/whisper-darija-stage1
Intended Uses
This model is suitable for:
- Automatic Speech Recognition (ASR)
- Moroccan Darija transcription
- Voice assistants
- Flight booking assistants
- Conversational AI
- Travel applications
Limitations
This model was fine-tuned only on Moroccan Darija speech related to flight booking.
Performance may decrease for:
- General conversations
- Medical vocabulary
- Legal vocabulary
- Noisy audio
- Strong regional accents not represented in the training data
Dataset
The model was trained on a custom Moroccan Darija speech dataset containing flight-related queries.
The dataset includes recordings covering:
- Departure cities
- Arrival cities
- Dates
- Passenger counts
- Flight reservations
- Flight search requests
Training Procedure
Hyperparameters
| Parameter | Value |
|---|
| Learning Rate | 5e-6 |
| Train Batch Size | 8 |
| Eval Batch Size | 8 |
| Gradient Accumulation | 2 |
| Total Batch Size | 16 |
| Warmup Steps | 40 |
| Max Training Steps | 200 |
| Weight Decay | 0.01 |
| FP16 | True |
| Gradient Checkpointing | True |
| Optimizer | AdamW |
| LR Scheduler | Linear |
Training Results
| Step | Training Loss | Validation Loss | WER |
|---|
| 50 | 1.5132 | 0.4115 | 30.17 |
| 100 | 0.1160 | 0.2923 | 25.51 |
| 150 | 0.0102 | 0.2779 | 23.28 |
| 200 | 0.0048 | 0.2775 | 23.46 |
Best checkpoint:
- Validation Loss: 0.2779
- WER: 23.28
Evaluation
The model achieved:
- Word Error Rate (WER): 23.28%
This represents a significant improvement over the base model for Moroccan Darija flight-related speech recognition.
Example Use Cases
The model can transcribe requests such as:
"بغيت نحجز رحلة من كازا لباريس."
"بغيت نمشي لطنجة نهار الجمعة."
Framework Versions
- Transformers
- PyTorch
- Datasets
- Tokenizers
