A fine-tuned CSM (Conditional Speech Model) for generating natural-sounding Nigerian English speech from text. This model specializes in producing TTS output with authentic Nigerian accent and pronunciation patterns.

Model Details

Model Description

This is a fine-tuned Text-to-Speech (TTS) model based on the CSM (Conditional Speech Model) architecture, specifically trained on Nigerian English audio data to generate natural-sounding speech with Nigerian accent and pronunciation patterns. The model uses LoRA (Low-Rank Adaptation) fine-tuning for efficient training and maintains the multi-speaker capabilities of the base model.

Developed by: Sam4rano
Model type: Text-to-Speech (TTS) / Conditional Speech Model
Language(s) (NLP): English (Nigerian variant)
License: Apache 2.0
Finetuned from model: unsloth/csm-1b

Model Sources

Repository: https://huggingface.co/Sam4rano/naija-csm-finetuned-model
Base Model: unsloth/csm-1b
Training Framework: Unsloth

Uses

Direct Use

This model can be used to generate Nigerian-accented English speech from text input. It's particularly useful for:

Creating TTS systems for Nigerian applications
Generating natural-sounding Nigerian English audio for educational content
Voice synthesis for Nigerian-focused applications and services

Downstream Use

Audiobook narration with Nigerian accent
Educational tools for Nigerian English pronunciation

Out-of-Scope Use

This model is not suitable for generating speech in languages other than Nigerian English
Should not be used for creating misleading, deceptive, or harmful audio content
Not recommended for real-time applications without proper optimization
Not suitable for other English variants or accents without additional fine-tuning

Bias, Risks, and Limitations

Accent Specificity: Trained primarily on Nigerian English, may not perform well for other English variants
Speaker Diversity: Quality may vary depending on the diversity of speakers in the training data
Audio Quality: Output quality depends on the quality and diversity of the training dataset
Computational Requirements: Requires GPU for optimal inference speed
Limited Training: Model was trained for only 60 steps as a demonstration - production use may require additional training

Recommendations

Users should be aware that this model is optimized for Nigerian English and may not generalize well to other accents or languages. Consider additional training with more data and epochs for improved performance.

How to Get Started with the Model

Use the code below to get started with the model.

python

from transformers import CsmForConditionalGeneration, AutoProcessor
import torch
import soundfile as sf

# Load the model and processor
model = CsmForConditionalGeneration.from_pretrained("Sam4rano/naija-csm-finetuned-model")
processor = AutoProcessor.from_pretrained("Sam4rano/naija-csm-finetuned-model")

# Generate speech
text = "Welcome to Nigeria, the giant of Africa!"
speaker_id = 0

# Method 1: Simple text input
inputs = processor(f"[{speaker_id}]{text}", add_special_tokens=True)
audio_values = model.generate(
    **inputs,
    max_new_tokens=125,  # 125 tokens ≈ 10 seconds of audio
    output_audio=True
)

# Convert to audio array and save
audio = audio_values[0].to(torch.float32).cpu().numpy()
sf.write("nigerian_speech.wav", audio, 24000)

# Method 2: Conversation format (recommended)
conversation = [
    {"role": str(speaker_id), "content": [{"type": "text", "text": text}]},
]

inputs = processor.apply_chat_template(
    conversation,
    tokenize=True,
    return_dict=True,
)

audio_values = model.generate(
    **inputs,
    max_new_tokens=125,
    output_audio=True
)

audio = audio_values[0].to(torch.float32).cpu().numpy()
sf.write("nigerian_speech.wav", audio, 24000)

# For voice consistency, provide audio context
sample_audio = your_reference_audio_array  # numpy array at 24kHz
sample_text = "Your reference text"

conversation_with_context = [
    {"role": str(speaker_id), "content": [
        {"type": "text", "text": sample_text},
        {"type": "audio", "path": sample_audio}
    ]},
    {"role": str(speaker_id), "content": [{"type": "text", "text": text}]},
]

inputs = processor.apply_chat_template(conversation_with_context, tokenize=True, return_dict=True)
audio_values = model.generate(**inputs, max_new_tokens=125, output_audio=True)

Training Details

Training Data

The model was trained on a custom Nigerian English audio dataset containing text-audio pairs with speaker information. The dataset was preprocessed to ensure consistent formatting and quality.

Data characteristics:

Language: Nigerian English
Audio format: 24kHz sampling rate
Speaker handling: Multi-speaker support with consistent speaker IDs
Text processing: Tokenized with special speaker tokens

Training Procedure

Preprocessing

The training data underwent several preprocessing steps:

Audio files resampled to 24kHz sampling rate
Maximum audio length: 240,001 samples (approximately 10 seconds)
Maximum text length: 256 tokens
Speaker IDs converted to consistent string format
Audio padding to maximum length for batch processing
Text padding with attention masks

Training Hyperparameters

Training regime: Mixed precision (fp16/bf16) with LoRA fine-tuning
Base model: unsloth/csm-1b
LoRA Configuration:
- LoRA rank (r): 32
- LoRA alpha: 32
- LoRA dropout: 0
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Learning rate: 2e-4
Batch size: 2 (per device)
Gradient accumulation steps: 4
Training steps: 60
Warmup steps: 5
Optimizer: AdamW 8-bit
Weight decay: 0.01
LR scheduler: Linear
Random seed: 3407

Speeds, Sizes, Times

Training time: Approximately 2-5 minutes on Tesla T4 GPU
Memory usage: ~8-12GB GPU memory during training
Model size: LoRA adapters only (~few MB), requires base model for full functionality
Hardware: Google Colab Tesla T4 GPU (16GB VRAM)

Evaluation

Testing Data, Factors & Metrics

Testing Data

Due to the demonstration nature of this training (60 steps), formal evaluation was limited to basic functionality testing and audio generation quality assessment.

Factors

Audio quality and naturalness
Speaker consistency
Pronunciation accuracy for Nigerian English
Text-to-speech alignment

Metrics

Qualitative audio assessment
Model loading and inference functionality
Generation capability verification

Results

The model successfully generates Nigerian-accented English speech and maintains the multi-speaker capabilities of the base model. However, extended training would be beneficial for production use.

Summary

This model serves as a proof-of-concept for Nigerian English TTS and demonstrates the effectiveness of LoRA fine-tuning for speech synthesis models.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: NVIDIA Tesla T4 GPU
Hours used: ~0.1 hours (2-5 minutes)
Cloud Provider: Google Colab
Compute Region: Variable
Carbon Emitted: Minimal due to short training time

Technical Specifications

Model Architecture and Objective

Architecture: CSM (Conditional Speech Model) - transformer-based text-to-speech model
Objective: Conditional generation of speech audio from text with speaker conditioning
Input: Text tokens + speaker ID
Output: Audio waveform at 24kHz sampling rate
Fine-tuning method: LoRA (Low-Rank Adaptation)

Compute Infrastructure

Training was conducted using the Unsloth framework for memory-efficient fine-tuning.

Hardware

Google Colab Tesla T4 GPU (16GB VRAM)
CUDA-compatible environment
Memory-optimized training with Unsloth

Software

Python 3.10+
PyTorch 2.0+
Transformers 4.52.3
Unsloth framework
Additional dependencies: soundfile, datasets, accelerate

Citation

BibTeX:

Glossary

CSM: Conditional Speech Model - A transformer-based architecture for text-to-speech synthesis
LoRA: Low-Rank Adaptation - An efficient fine-tuning method that updates only a small subset of model parameters
TTS: Text-to-Speech - Technology that converts written text into spoken audio
Nigerian English: The variety of English spoken in Nigeria, with distinct pronunciation and linguistic features

Model Card Authors

Sam4rano

Model Card Contact

For questions or issues regarding this model, please create an issue in the model repository or contact through Hugging Face.