This is a fine-tuned version of the Facebook MMS (Massively Multilingual Speech) model for Swahili Text-to-Speech (TTS). The model was fine-tuned to improve Swahili pronunciation and performance using custom audio datasets.
Model Details
Model Name: Swahili MMS TTS - Finetuned
Languages Supported: Swahili
Base Model: Facebook MMS
Use Case: Text-to-Speech for Swahili language, suitable for generating high-quality speech from text.
Training Details
The fine-tuning process was done using a custom dataset of Swahili voice samples to improve the fluency and accuracy of the original MMS model in Swahili. This resulted in enhanced pronunciation and natural-sounding speech for Swahili.
You can check out the code and process used in the fine-tuning by visiting the GitHub repository.
How to Use
You can load and use the model directly from the Hugging Face model hub using either the pipeline API or by manually downloading the model and tokenizer.
1. Download and Run the Model Directly
You can also download the model and tokenizer manually and run the text-to-speech pipeline without the Hugging Face pipeline helper. Here's how:
python
1import torch
2import numpy as np
3import scipy.io.wavfile
4from transformers import VitsModel, AutoTokenizer
567device = torch.device("cuda"if torch.cuda.is_available()else"cpu")8model_name ="Benjamin-png/swahili-mms-tts-finetuned"9text ="Habari, karibu kwenye mfumo wetu wa kusikiliza kwa Kiswahili."10audio_file_path ="swahili_speech.wav"1112# Load model and tokenizer dynamically based on the provided model name13model = VitsModel.from_pretrained(model_name).to(device)14tokenizer = AutoTokenizer.from_pretrained(model_name)1516# Step 1: Tokenize the input text17inputs = tokenizer(text, return_tensors="pt").to(device)1819# Step 2: Generate waveform20with torch.no_grad():21 output = model(**inputs).waveform
2223# Step 3: Convert PyTorch tensor to NumPy array24output_np = output.squeeze().cpu().numpy()2526# Step 4: Write to WAV file27scipy.io.wavfile.write(audio_file_path, rate=model.config.sampling_rate, data=output_np)
2. Using the pipeline API
python
1from transformers import pipeline
23# Load the fine-tuned model4tts = pipeline("text-to-speech", model="Benjamin-png/swahili-mms-tts-finetuned")56# Generate speech from text7speech = tts("Habari, karibu kwenye mfumo wetu wa kusikiliza kwa Kiswahili.")
Saving and Playing the Audio
To save and play the audio, you can use the same methods mentioned above:
Saving the Audio
python
1import soundfile as sf
23# Save the audio as a WAV file4sf.write("swahili_speech.wav", output_np, model.config.sampling_rate)
Playing the Audio
You can play the audio using pydub:
python
1from pydub import AudioSegment
2from pydub.playback import play
34# Load and play the generated audio5audio = AudioSegment.from_wav("swahili_speech.wav")6play(audio)
If you're interested in reproducing the fine-tuning process or using the model for similar purposes, you can check out the Google Colab notebook that outlines the entire process:
The notebook includes detailed steps on how to fine-tune the MMS model for Swahili TTS.
GitHub Repository
For further exploration and code snippets, visit the GitHub repository where you’ll find additional scripts, datasets, and instructions for customizing the model.
License
This project is licensed under the terms of the Apache License 2.0.
