A high-speed, high-fidelity Text-to-Speech model optimized for real-time conversational AI applications.
Overview
KaniTTS uses a two-stage pipeline combining a large language model with an efficient audio codec for exceptional speed and audio quality. The architecture generates compressed token representations through a backbone LLM, then rapidly synthesizes waveforms via neural audio codec, achieving extremely low latency.
Lower RTF is better (< 1.0 means faster than real-time). Benchmarks conducted on Vast AI.
Quickstart: Install from PyPI & Run Inference
It’s a lightweight so you can install, load a model, and speak in minutes.
Designed for quick starts and simple workflows—no heavy setup, just pip install and run.
More detailes...
Install
bash
1pip install kani-tts
2pip install -U "transformers==4.57.1"# for LFM2 !!!
Quick Start
python
1from kani_tts import KaniTTS
23model = KaniTTS('nineninesix/kani-tts-400m-en')45# Generate audio from text6audio, text = model("Hello, world!")78# Save to file (requires soundfile)9model.save_audio(audio,"output.wav")
Working with Multi-Speaker Models
This model support multiple speakers. You can check if your model supports speakers and select a specific voice:
python
1from kani_tts import KaniTTS
23model = KaniTTS('nineninesix/kani-tts-400m-en')45# Check if model supports multiple speakers6print(f"Model type: {model.status}")# 'singlspeaker' or 'multispeaker'78# Display available speakers (pretty formatted)9model.show_speakers()1011# Or access the speaker list directly12print(model.speaker_list)# ['andrew', 'katie']1314# Generate audio with a specific speaker15audio, text = model("Hello, world!", speaker_id="andrew")
Custom Configuration
python
1from kani_tts import KaniTTS
23model = KaniTTS(4'your-model-name',5 temperature=0.7,# Control randomness (default: 1.0)6 top_p=0.9,# Nucleus sampling (default: 0.95)7 max_new_tokens=2000,# Max audio length (default: 1200)8 repetition_penalty=1.2,# Prevent repetition (default: 1.1)9 suppress_logs=True,# Suppress library logs (default: True)10 show_info=True,# Show model info on init (default: True)11)1213audio, text = model("Your text here")
Playing Audio in Jupyter Notebooks
You can listen to generated audio directly in Jupyter notebooks or IPython:
python
1from kani_tts import KaniTTS
2from IPython.display import Audio as aplay
34model = KaniTTS('your-model-name')5audio, text = model("Hello, world!")67# Play audio in notebook8aplay(audio, rate=model.sample_rate)
Holy fu- Oh my God! Don't you understand how dangerous it is?
Colleges of Oxford, Cambridge, Durham and the University of the Highlands and Islands UHI are 'listed bodies', as bodies that appear to the Secretary of State to be constituent colleges, schools, halls or other institutions of a university.
A joyful flock of sparrows chirped merrily in the old oak tree outside my window this morning.
Darlin', I still ain't feelin' so well. I'm goin' to bed.
Use Cases
Conversational AI: Real-time speech for chatbots and virtual assistants
Edge/Server Deployment: Resource-efficient inference on affordable hardware
Accessibility: Screen readers and language learning applications
Research: Fine-tuning for specific voices, accents, or emotions
Limitations
Performance degrades with inputs exceeding 15 seconds (need to use sliding window chunking)
Limited expressivity without fine-tuning for specific emotions
May inherit biases from training data in prosody or pronunciation
Optimized primarily for English; other languages may require additional training
Optimization Tips
Multilingual Performance: Continually pretrain on target language datasets and fine-tune NanoCodec
Batch Processing: Use batches of 8-16 for high-throughput scenarios
Hardware: Optimized for NVIDIA Blackwell architecture GPUs
Illegal content or harmful, threatening, defamatory, or obscene material
Hate speech, harassment, or incitement of violence
Generating false or misleading information
Impersonating individuals without consent
Malicious activities such as spamming, phishing, or fraud
By using this model, you agree to comply with these restrictions and all applicable laws.
Contact
Have a question, feedback, or need support? Please fill out our contact form and we'll get back to you as soon as possible.
Citation
@inproceedings{emilialarge,
author={He, Haorui and Shang, Zengqiang and Wang, Chaoren and Li, Xuyuan and Gu, Yicheng and Hua, Hua and Liu, Liwei and Yang, Chen and Li, Jiaqi and Shi, Peiyang and Wang, Yuancheng and Chen, Kai and Zhang, Pengyuan and Wu, Zhizheng},
title={Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation},
booktitle={arXiv:2501.15907},
year={2025}
}
@article{emonet_voice_2025,
author={Schuhmann, Christoph and Kaczmarczyk, Robert and Rabby, Gollam and Friedrich, Felix and Kraus, Maurice and Nadi, Kourosh and Nguyen, Huu and Kersting, Kristian and Auer, Sören},
title={EmoNet-Voice: A Fine-Grained, Expert-Verified Benchmark for Speech Emotion Detection},
journal={arXiv preprint arXiv:2506.09827},
year={2025}
}
🚀 If you find these models useful
Help me test my AI-Powered Quantum Network Monitor Assistant with quantum-ready security checks:
The full Open Source Code for the Quantum Network Monitor Service available at my github repos ( repos with NetworkMonitor in the name) : Source Code Quantum Network Monitor. You will also find the code I use to quantize the models if you want to do it yourself GGUFModelBuilder
💬 How to test:
Choose an AI assistant type:
TurboLLM (GPT-4.1-mini)
HugLLM (Hugginface Open-source models)
TestLLM (Experimental CPU-only)
What I’m Testing
I’m pushing the limits of small open-source models for AI network monitoring, specifically:
Function calling against live network services
How small can a model go while still handling:
Automated Nmap security scans
Quantum-readiness checks
Network Monitoring tasks
🟡 TestLLM – Current experimental model (llama.cpp on 2 CPU threads on huggingface docker space):
✅ Zero-configuration setup
⏳ 30s load time (slow inference but no API costs) . No token limited as the cost is low.
🔧 Help wanted! If you’re into edge-device AI, let’s collaborate!
Other Assistants
🟢 TurboLLM – Uses gpt-4.1-mini :
**It performs very well but unfortunatly OpenAI charges per token. For this reason tokens usage is limited.
Create custom cmd processors to run .net code on Quantum Network Monitor Agents
Real-time network diagnostics and monitoring
Security Audits
Penetration testing (Nmap/Metasploit)
🔵 HugLLM – Latest Open-source models:
🌐 Runs on Hugging Face Inference API. Performs pretty well using the lastest models hosted on Novita.
💡 Example commands you could test:
"Give me info on my websites SSL certificate"
"Check if my server is using quantum safe encyption for communication"
"Run a comprehensive security audit on my server"
'"Create a cmd processor to .. (what ever you want)" Note you need to install a Quantum Network Monitor Agent to run the .net code on. This is a very flexible and powerful feature. Use with caution!
Final Word
I fund the servers used to create these model files, run the Quantum Network Monitor service, and pay for inference from Novita and OpenAI—all out of my own pocket. All the code behind the model creation and the Quantum Network Monitor project is open source. Feel free to use whatever you find helpful.
If you appreciate the work, please consider buying me a coffee ☕. Your support helps cover service costs and allows me to raise token limits for everyone.
I'm also open to job opportunities or sponsorship.
