MagpieTTS is an end-to-end multilingual neural text-to-speech model that synthesizes speech using 5 English speaker voices — Aria, Jason, Leo, Sofia, and John Van Stan — across 12 languages: Arabic (ar), Chinese (zh), English (en), French (fr), German (de), Hindi (hi), Italian (it), Japanese (ja), Korean (ko), Portuguese (pt), Spanish (es), and Vietnamese (vi). The model adopts a transformer encoder–decoder architecture that autoregressively predicts discrete audio codec tokens, using multi-codebook prediction (typically 8 codebooks) with frame stacking (factor = 2) and a local transformer for fine-grained refinement of high-fidelity audio. To improve robustness and controllability, training incorporates attention priors for stable text-to-audio alignment, classifier-free guidance (CFG) for stronger conditioning, and Group Relative Policy Optimization (GRPO) for preference-aligned generation. At inference time, MagpieTTS supports batched synthesis of complete utterances as well as long-form generation of extended text via a sliding-window mechanism; the predicted codec tokens are then decoded into speech waveforms by a frozen pretrained audio codec model (NanoCodec). This release also removed zero-shot voice-cloning capability for security reasons, and added IPA grapheme-to-phoneme (G2P) support for custom dictionaries and code-switching, and updated G2P support for English-to-Katakana code-switching.
This model is ready for commercial use.
Key Features
Multilingual Support — Synthesizes natural speech across all 12 supported languages with consistent speaker identity.
Expressive Voices — Multiple voice options with emotional tones and gender variations including 4 proprietary voices and 1 public voice.
Text Normalization — Built-in text normalization for handling numbers, abbreviations, and special characters for all 12 languages.
Efficient High-Fidelity Decoding — A local transformer with frame stacking (factor = 2) performs multi-codebook refinement on stacked frames, improving audio quality while reducing sequence length for faster generation.
MagpieTTS is for developers, researchers, and product teams building multilingual speech applications that need consistent speaker voices across 12 languages. Typical applications include cascade voice agents, audiobook and content narration, accessibility tools, dubbing and localization pipelines, and interactive media. IPA grapheme-to-phoneme (G2P) support for custom dictionaries and code-switching (including English-to-Katakana) also enables mixed-language content and domain-specific pronunciation.
MagpieTTS acts as a dedicated speech-generation layer that plugs into existing AI pipelines without changing upstream language models or downstream audio handling. In cascade voice-agent setups, it converts Large Language Model (LLM) text into natural, real-time speech for user playback. It can also replace or extend existing NVIDIA TTS integrations when multilingual coverage from a single unified model is required.
Architecture Type: Transformer Encoder, Transformer Decoder, Local Transformer, and Feedforward Layers
MagpieTTS Model Architecture
Figure 1: MagpieTTS Model Architecture
Network Architecture:
Causal Transformer Encoder with 6 layers, learnable positional encoder of length 2048, and 1 Layer Normalization output layer.
Causal Transformer Decoder with 12 layers, learnable positional encoder of length 2048, and 1 Layer Normalization output layer.
Local Transformer for multi-codebook refinement, operating on stacked frames with a frame stacking factor of 2 to improve audio quality and reduce sequence length.
Number of model parameters:3.64 x 10^8 (364M parameters)
Inputs & Outputs
Input Type(s): Text Input Format: String Input Parameters: One-Dimensional (1D) Other Properties Related to Input: Text input is UTF-8 encoded; text normalization is required.
Output Type(s): Audio Output Format: WAV Output Parameters: One-Dimensional (1D) Other Properties Related to Output: Mono, PCM-encoded 16 bit audio; sampling rate of 22.05 kHz; Audio output with dimensions (B x T), where B is batch size and T is time dimension.
Note: Voice IDs and supported sample rates follow the deployed NIM. Use --list-voices against this endpoint, or see the API Reference on the Magpie TTS Multilingual page.
Run MagpieTTS locally with NeMo-Speech.cpp
NeMo-Speech.cpp provides a
lightweight native C++ runtime for running this model locally. This v2602
GGUF uses the tokenizer assets contained in the original .nemo archive and
the companion NanoCodec decoder.
To train, fine-tune or perform TTS with this model, you will need to install NVIDIA NeMo Speech. We recommend you install it after you've installed latest PyTorch version and Python version ≥ 3.10.12.
The model is available for use in the NVIDIA NeMo Speech Framework, and can be used as a pre-trained checkpoint for inference or for fine-tuning on another dataset.
Two inference paths are available: Method 1 (single utterance) and Method 2 (batch inference and evaluation).
Method 1 — Single utterance inference
Synthesize one (text, language) pair at a time. Text normalization can be applied for all 12 languages.
Load the open-source MagpieTTS checkpoint from Hugging Face and call the model's do_tts(transcript: str, language: str, apply_TN: bool, use_cfg: bool, speaker_index: int) method. This returns the generated audio and the length of the audio.
python
1from nemo.collections.tts.models import MagpieTTSModel
23speaker_map ={4"Aria":0,5"Jason":1,6"John":2,7"Leo":3,8"Sofia":4,9}10transcript ="Hello world from NeMo Text to Speech."11language ="en"12speaker ="Sofia"13speaker_idx = speaker_map[speaker]1415# Load the latest checkpoint (from the `main` branch).16model = MagpieTTSModel.from_pretrained("nvidia/magpie_tts_multilingual_357m")17audio, audio_len = model.do_tts(transcript, language=language, apply_TN=False, speaker_index=speaker_idx)1819# To apply custom phoneme customization in supported languages like English20# Surround the IPA string in a '|' character and add a space token between each IPA character.21ipa_transcript ="Hello world from | ˈ n ɛ m o ʊ | Text to Speech."22audio, audio_len = model.do_tts(ipa_transcript, language=language, apply_TN=False, speaker_index=speaker_idx)
Choosing a model version
from_pretrained(...) always loads the latest checkpoint from the main branch. To load a specific release instead, download the .nemo file for that version tag and restore it with restore_from(...). Available tags: v2607 (latest, on main), v2602, and v2512.
python
1from huggingface_hub import hf_hub_download
2from nemo.collections.tts.models import MagpieTTSModel
34# Pin a specific release by its tag (branch, tag, or commit hash).5model_path = hf_hub_download(6 repo_id="nvidia/magpie_tts_multilingual_357m",7 filename="magpie_tts_multilingual_357m.nemo",8 revision="v2602",9)10model = MagpieTTSModel.restore_from(model_path)
Method 2 — Batch inference and evaluation
Run batch inference and optional evaluation with examples/tts/magpietts_inference.py. The script supports:
Batch inference from .nemo files or .ckpt checkpoints
Optional evaluation with metrics — Character Error Rate (CER), Speaker Similarity (SSIM), and UTMOSv2
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
Number of data items in training set: ~54.3k hours
Data modalities: text and audio
Nature of the content: Audiobooks, Daily Conversations, podcast interviews, News, Twitter, Youtube
Languages: 12 languages (Arabic (ar), Chinese (zh), English (en), French (fr), German (de), Hindi (hi), Italian (it), Japanese (ja), Korean (ko), Portuguese (pt), Spanish (es), Vietnamese (vi))
Evaluation data spans all 12 supported languages with the same modalities, content types, and collection method as the training data.
Evaluation
We report two metrics on per-language held-out test sets:
CER — Character Error Rate (%), lower is better.
SSIM — speaker similarity between prediction and context, higher is better.
Language / dataset
CER↓ (%)
SSIM↑ (pred-context)
v2602
v2607
v2602
v2607
Arabic (ar)
—
1.62
—
0.806
Chinese (zh)
—
3.17
—
0.833
English (en)
0.34
0.37
0.835
0.822
English (en) accented
—
0.34
—
0.746
French (fr)
2.70
1.54
0.703
0.747
German (de)
0.66
0.80
0.626
0.742
Hindi (hi)
—
1.23
—
0.788
Italian (it)
—
2.19
—
0.773
Japanese (ja)
—
1.40
—
0.775
Korean (ko)
—
2.69
—
0.807
Portuguese (pt)
—
2.91
—
0.753
Spanish (es)
1.14
0.60
0.715
0.793
Vietnamese (vi)
—
0.59
—
0.725
Technical Limitations & Mitigations
There are two modes of inference, namely, standard and long-form. In standard mode, this model can generate up to 20 seconds of speech at a time in any of the 12 supported languages. In long-form mode, the model performs optimally when the input text contains punctuation and capitalization. The model was trained on a mix of publicly available speech datasets and internally recorded datasets in 12 languages. As a result, it is not suitable for speech generation in any language other than the 12 languages mentioned. We have removed zero-shot capabilities of this model for this release. Text normalization is required.
Ethical Considerations
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.
-lightgrey#model-badge){kind=icon}