You can deploy this model and start earning money today!

oneplanettechnologies/
my-big-model

Name: my-big-model
Availability: InStock

Views

No views yet

Voicecraft: The Art of Text-to-Speech

A complete, production-ready Text-to-Speech (TTS) system that crafts human-like voices from text. This system uses Tacotron2 for mel-spectrogram generation and HiFiGAN for high-quality neural vocoding.

🎯 Project Overview

Voicecraft is a comprehensive TTS system developed by students of Dhemaji Engineering College, Computer Science and Engineering Department, under the guidance of Barga Deori Sir.

Team Members:

Abinash Dutta
Arindom Bordoloi
Bakhtiar Ahmed
Chinmoy Mahanta
Mumon Saikia
Preety Rani Boruah

✨ Features

Core Capabilities

Text Normalization: Handles numbers, abbreviations, and special symbols
G2P Conversion: Grapheme-to-Phoneme with CMU dictionary and rule-based fallback
Prosody Prediction: Predicts pitch, duration, rhythm, stress, and intonation
Tacotron2: State-of-the-art mel-spectrogram generation with location-sensitive attention
HiFiGAN Vocoder: High-quality neural vocoder for natural waveform synthesis
Audio Preprocessing: Silence trimming, noise reduction, normalization
Evaluation Metrics: MCD, Spectral Distortion, and MOS helper

User Interface

Gradio Web Interface: Beautiful, responsive GUI
Real-time Synthesis: Fast inference with GPU support
Visualization: Mel-spectrogram display
Phoneme Display: Shows G2P conversion results
Audio Playback & Download: Listen and save generated audio

📁 Project Structure

voicecraft/
├── datasets/              # Dataset handling
│   ├── download_ljspeech.py
│   └── prepare_dataset.py
├── preprocessing/         # Text and audio preprocessing
│   ├── text_normalizer.py
│   └── audio_processor.py
├── g2p/                  # Grapheme-to-Phoneme conversion
│   └── g2p_converter.py
├── prosody/              # Prosody prediction
│   └── prosody_predictor.py
├── tacotron2/            # Tacotron2 acoustic model
│   ├── model.py
│   └── train.py
├── vocoder/              # Vocoders (HiFiGAN, WaveGlow)
│   ├── hifigan.py
│   └── train.py
├── utils/                # Utility functions
│   ├── audio_utils.py
│   ├── collate_fn.py
│   └── evaluation.py
├── deployment/           # Deployment files
│   ├── app.py            # Gradio GUI
│   ├── requirements.txt
│   └── colab_deploy.ipynb
├── future/               # Future expansion modules
│   ├── assamese_tts.py
│   ├── emotional_tts.py
│   ├── whisper_transcription.py
│   └── mobile_inference.py
└── README.md

🚀 Quick Start

Installation

Clone the repository (or download the code):

bash

git clone <repository-url>
cd voicecraft

Install dependencies:

pip install -r deployment/requirements.txt

Download datasets (optional, for training):

bash

python voicecraft/datasets/download_ljspeech.py
python voicecraft/datasets/prepare_dataset.py data/LJSpeech-1.1

Running the GUI

Start the Gradio interface:

python voicecraft/deployment/app.py

Open your browser to http://localhost:7860
Use the interface:
- Enter text in the text box
- Click "Paste Intro Paragraph" for demo text
- Click "Generate Speech" to synthesize
- Listen to the audio and view phonemes/spectrogram

📊 Datasets

Primary Dataset: LJSpeech 1.1

13,100 text-audio pairs
24 hours of clean speech
22.05 kHz WAV format
Used for Tacotron2 + HiFiGAN training

Download and Prepare

bash

# Download LJSpeech
python voicecraft/datasets/download_ljspeech.py

# Prepare dataset with train/val/test splits
python voicecraft/datasets/prepare_dataset.py data/LJSpeech-1.1

The dataset will be split into:

90% training (11,790 samples)
5% validation (655 samples)
5% test (655 samples)

Secondary Dataset (Optional)

For experimental purposes, a small custom dataset (100-500 samples) can be used for GlowTTS experimentation.

🏗️ System Architecture

Complete Pipeline

Text Preprocessing
- Text normalization (numbers → words, abbreviation expansion)
- Special symbol cleaning
- Whitespace normalization
G2P Conversion
- CMU dictionary lookup
- Rule-based fallback for unknown words
- Phoneme sequence generation
Prosody Prediction
- Pitch prediction
- Duration estimation
- Rhythm patterns
- Stress patterns
- Intonation contours
Tacotron2 Acoustic Model
- Encoder: Character embeddings + CNN + BiLSTM
- Location-Sensitive Attention: Monotonic alignment
- Decoder: Attention RNN + Decoder RNN + Postnet
- Output: Mel-spectrogram
HiFiGAN Vocoder
- Multi-period discriminator
- Generator with residual blocks
- Fast inference
- High-quality waveform synthesis
Post-processing
- Audio normalization
- Silence trimming
- Quality enhancement

🔬 Training

Tacotron2 Training

bash

python voicecraft/tacotron2/train.py \
    --train_data data/processed/train \
    --val_data data/processed/val \
    --checkpoint_dir checkpoints/tacotron2 \
    --log_dir logs/tacotron2 \
    --num_epochs 100 \
    --batch_size 32 \
    --lr 1e-3 \
    --use_amp \
    --save_interval 1000

Training Parameters:

Steps: 150k-200k steps recommended
Optimizer: Adam (lr=1e-3, weight_decay=1e-6)
Scheduler: ExponentialLR (gamma=0.999)
Mixed Precision: AMP enabled
Multi-GPU: Optional (DDP support)

Loss Functions:

L1 loss (mel-spectrogram)
MSE loss (mel-spectrogram)
Binary cross-entropy (gate prediction)

HiFiGAN Training

bash

python voicecraft/vocoder/train.py \
    --train_data data/processed/train \
    --checkpoint_dir checkpoints/hifigan \
    --log_dir logs/hifigan \
    --num_epochs 100 \
    --batch_size 16 \
    --lr_g 2e-4 \
    --lr_d 2e-4 \
    --use_amp \
    --save_interval 1000

Training Parameters:

Steps: 250k steps recommended
Optimizer: AdamW (betas=(0.8, 0.99))
Losses: Generator loss, discriminator loss, feature matching loss

📈 Evaluation

Metrics

MCD (Mel Cepstral Distortion)
- Lower is better
- Typical range: 4-8 dB
Spectral Distortion
- Lower is better
- Measures mel-spectrogram quality
MOS Helper
- Estimated quality indicators
- SNR, spectral centroid, etc.

Usage

python

from voicecraft.utils.evaluation import compute_mcd, compute_spectral_distortion

mcd = compute_mcd(mel_pred, mel_target)
sd = compute_spectral_distortion(mel_pred, mel_target)

🌐 Deployment

HuggingFace Spaces

Create a new Space on HuggingFace
Upload files:
- voicecraft/deployment/app.py
- voicecraft/deployment/requirements.txt
- All voicecraft/ source code
- Model checkpoints (or use model loading from URLs)
Configure Space:
- Hardware: CPU or GPU
- Python SDK
The app will automatically:
- Install dependencies
- Load models on startup
- Provide public URL

Google Colab

Upload the notebook:
- voicecraft/deployment/colab_deploy.ipynb
Run cells:
- Install dependencies
- Load models
- Start Gradio interface
Create public tunnel:

python
1from pyngrok import ngrok 2public_url = ngrok.connect(7860) 3print(f"Public URL: {public_url}")
Share the link for public access

Local Deployment

bash

# Install dependencies
pip install -r voicecraft/deployment/requirements.txt

# Run the app
python voicecraft/deployment/app.py

The app will be available at http://localhost:7860

🎨 GUI Features

The Gradio interface includes:

Text Input: Multi-line text box
"Paste Intro Paragraph" Button: Automatically fills demo text
"Generate Speech" Button: Synthesizes audio
"Clear" Button: Resets all fields
Audio Output: Play generated speech
Phoneme Display: Shows G2P conversion
Mel-Spectrogram Image: Visual representation
Download Audio: Save generated files
Presentation Mode: Toggle for enhanced visualization (stub)

Special Feature: Paste Intro Paragraph

Click the "📋 Paste Intro Paragraph" button to automatically fill the text box with:

Hello everyone, hope you are well. We are students of Dhemaji Engineering College, Computer Science and Engineering Department, and our names are: Abinash Dutta, Arindom Bordoloi, Bakhtiar Ahmed, Chinmoy Mahanta, Mumon Saikia, Preety Rani Boruah. We are developing the project "Voicecraft: The Art of Text-to-Speech – Crafting Human-Like Voices from Text" under the guidance of Barga Deori Sir. Thank you.

🔮 Future Expansions

Planned Features

Assamese TTS (future/assamese_tts.py)
- Assamese language support
- Assamese-specific preprocessing
- Assamese G2P conversion
Emotional TTS (future/emotional_tts.py)
- Happy, Sad, Angry emotions
- Emotion-conditioned synthesis
- Prosody modification
Whisper Transcription (future/whisper_transcription.py)
- Speech-to-text integration
- Automatic transcription
- Multi-language support
Mobile Inference (future/mobile_inference.py)
- Model quantization
- ONNX/TFLite export
- Lightweight inference

📚 Documentation

Module Documentation

Dataset Handling: See voicecraft/datasets/
Preprocessing: See voicecraft/preprocessing/
G2P: See voicecraft/g2p/
Models: See voicecraft/tacotron2/ and voicecraft/vocoder/
Deployment: See voicecraft/deployment/

API Usage

python

from voicecraft.preprocessing.text_normalizer import normalize_text
from voicecraft.g2p.g2p_converter import text_to_phonemes
from voicecraft.tacotron2.model import Tacotron2
from voicecraft.vocoder.hifigan import HiFiGAN

# Normalize text
text = "Hello, it's 9:30 a.m. on Jan. 15, 2023."
normalized = normalize_text(text)

# Convert to phonemes
phonemes = text_to_phonemes(normalized)

# Load models
tacotron2 = Tacotron2()
hifigan = HiFiGAN()

# Generate speech
text_seq = text_to_sequence(normalized)
mel = tacotron2.infer(text_seq)
audio = hifigan.infer(mel)

🛠️ Development

Adding New Features

Text Processing: Extend preprocessing/text_normalizer.py
G2P: Add rules to g2p/g2p_converter.py
Models: Modify architectures in tacotron2/ or vocoder/
GUI: Update deployment/app.py

Testing

bash

# Test individual modules
python voicecraft/preprocessing/text_normalizer.py
python voicecraft/g2p/g2p_converter.py

📝 License

[Add your license here]

👥 Credits

Institution: Dhemaji Engineering College
Department: Computer Science and Engineering
Supervisor: Barga Deori Sir
Developers: Voicecraft Team

🙏 Acknowledgments

LJSpeech dataset creators
Tacotron2 authors
HiFiGAN authors
CMU Pronouncing Dictionary
Open source community

📞 Contact

For questions, issues, or contributions, please contact the development team.

Voicecraft: The Art of Text-to-Speech - Crafting Human-Like Voices from Text

oneplanettechnologies/my-big-model

Voicecraft: The Art of Text-to-Speech

🎯 Project Overview

✨ Features

Core Capabilities

User Interface

📁 Project Structure

🚀 Quick Start

Installation

Running the GUI

📊 Datasets

Primary Dataset: LJSpeech 1.1

Download and Prepare

Secondary Dataset (Optional)

🏗️ System Architecture

Complete Pipeline

🔬 Training

Tacotron2 Training

HiFiGAN Training

📈 Evaluation

Metrics

Usage

🌐 Deployment

HuggingFace Spaces

Google Colab

Local Deployment

🎨 GUI Features

Special Feature: Paste Intro Paragraph

🔮 Future Expansions

Planned Features

📚 Documentation

Module Documentation

API Usage

🛠️ Development

Adding New Features

Testing

📝 License

👥 Credits

🙏 Acknowledgments

📞 Contact

oneplanettechnologies/
my-big-model