MioCodec-25Hz-44.1kHz: High-Fidelity 44.1kHz Neural Audio Codec for Efficient Spoken Language Modeling
MioCodec-25Hz-44.1kHz is a high-fidelity neural audio codec designed for efficient spoken language modeling. Based on the
Kanade-Tokenizer implementation, MioCodec extends the capabilities to 44.1 kHz sampling rates, providing superior audio quality while maintaining a very low token rate.
For a lightweight and faster alternative, see
MioCodec-25Hz-24kHz.
🌟 Overview
MioCodec decomposes speech into two distinct components:
- Content Tokens: Discrete representations that primarily capture linguistic information and phonetic content ("what" is being said) at a low frame rate (25 Hz).
- Global Embeddings: A continuous vector representing broad acoustic characteristics ("how")—including speaker identity, recording environment, and microphone traits.
By disentangling these elements, MioCodec is ideal for Spoken Language Modeling.
Key features
- High-Resolution: Supports 44.1 kHz audio (compared to the standard 24 kHz in Kanade).
- Ultra-Low Bitrate: Achieves high-fidelity reconstruction at only 341 bps.
- End-to-End Optimization: Unlike original two-stage approaches, the codec and vocoder are jointly fine-tuned to minimize waveform artifacts and jitter.
📊 Model Comparison
| Model | Token Rate | Vocab Size | Bit Rate | Sample Rate | SSL Encoder | Vocoder | Parameters | Highlights |
|---|
| MioCodec-25Hz-44.1kHz | 25 Hz | 12,800 | 341 bps | 44.1 kHz | WavLM-base+ | MioVocoder (Jointly Tuned) | 118M (w/o vocoder) | High-quality, high sample rate |
| MioCodec-25Hz-24kHz | 25 Hz | 12,800 | 341 bps | 24 kHz | WavLM-base+ | - (iSTFTHead) | 132M | Lightweight, fast inference |
| kanade-25hz | 25 Hz | 12,800 | 341 bps | 24 kHz | WavLM-base+ | Vocos 24kHz | 118M (w/o vocoder) | Original 25Hz model |
| kanade-12.5hz | 12.5 Hz | 12,800 | 171 bps | 24 kHz | WavLM-base+ | Vocos 24kHz | 120M (w/o vocoder) | Original 12.5Hz model |
🚀 Quick Start
Installation
1# Install via pip
2pip install git+https://github.com/Aratako/MioCodec
3
4# Or using uv
5uv add git+https://github.com/Aratako/MioCodec
6
Basic Inference
Basic usage for encoding and decoding audio:
1from miocodec import MioCodec, load_audio
2import soundfile as sf
3
4# 1. Load model
5model = MioCodec.from_pretrained("Aratako/MioCodec-25Hz-44.1kHz").eval().cuda()
6
7# 2. Load audio
8waveform = load_audio("input.wav", sample_rate=model.config.sample_rate).cuda()
9
10# 3. Encode Audio
11features = model.encode(waveform)
12
13# 4. Decode to Waveform
14resynth = model.decode(features=features)
15
16# 5. Save
17sf.write("output.wav", resynth.cpu().numpy(), samplerate=model.config.sample_rate)
Voice Conversion (Zero-shot)
MioCodec allows you to swap speaker identities by combining the content tokens of a source with the global embedding of a reference.
1source = load_audio("source_content.wav", sample_rate=model.config.sample_rate).cuda()
2reference = load_audio("target_speaker.wav", sample_rate=model.config.sample_rate).cuda()
3
4# Perform conversion
5vc_wave = model.voice_conversion(source, reference)
6sf.write("converted.wav", vc_wave.cpu().numpy(), samplerate=model.config.sample_rate)
🏗️ Training Methodology
To achieve high-fidelity 44.1kHz reconstruction, MioCodec was trained in three phases. Phase 1 and 2 strictly follow the
original Kanade paper to establish feature alignment and spectral sharpness, while Phase 3 introduces a novel end-to-end waveform refinement stage.
Phase 1: Feature Alignment
This phase corresponds to the "Main Training Phase" described in the original paper. The model is trained to minimize both Mel-spectrogram loss and SSL feature reconstruction loss (using WavLM-base+). The vocoder is not utilized; the loss is computed directly on the predicted mel-spectrograms.
Phase 2: Adversarial Alignment
Following the "GAN Post-Training" phase of the original paper, we introduce adversarial training to sharpen the spectrograms. In this stage, the content branch is frozen, and only the decoder and global branch are updated. The model is trained using Mel-spectrogram loss combined with GAN losses (Adversarial + Feature Matching) applied in the mel domain.
Phase 3: End-to-End Waveform Refinement
To address residual artifacts such as jitter or tremor often found in mel-only training, we introduced a third phase that shifts the domain to raw waveforms.
In this phase, Vocoder is unfrozen, allowing the Codec Decoder and Vocoder to be fine-tuned jointly in an end-to-end manner. Similar to Phase 2, the content branch remains frozen. The training objective minimizes waveform artifacts using objectives adapted from
XCodec2 and
Inworld TTS-1, with specific parameters tuned for 44.1 kHz:
- Multi-Resolution Mel Spectrogram Loss: Using window lengths of
[32, 64, 128, 256, 512, 1024, 2048, 4096].
- Multi-Period Discriminator (MPD): Using periods of
[2, 3, 5, 7, 11, 17, 23, 37].
- Multi-Scale STFT Discriminator (MS-STFTD): Using FFT sizes of
[216, 348, 568, 920, 1494, 2414, 3908, 6328].
- RMS Loss: Adopted from Inworld TTS-1 to stabilize energy and volume.
📚 Training Data
The training datasets are listed below:
| Language | Approx. Hours | Dataset | Used in Phases |
|---|
| Japanese | ~15,000h | Various public HF datasets | 1, 2, 3 |
| English | ~500h | Libriheavy-HQ | 1, 2, 3 |
| English | ~4000h | MLS-Sidon | 1, 2 |
| English | ~9000h | HiFiTTS-2 | 3 |
| German | ~1,950h | MLS-Sidon | 1, 2 |
| Dutch | ~1,550h | MLS-Sidon | 1, 2 |
| French | ~1,050h | MLS-Sidon | 1, 2 |
| Spanish | ~900h | MLS-Sidon | 1, 2 |
| Italian | ~240h | MLS-Sidon | 1, 2 |
| Portuguese | ~160h | MLS-Sidon | 1, 2 |
| Polish | ~100h | MLS-Sidon | 1, 2 |
📜 Acknowledgements
- Codec Architecture: Based on the brilliant work of kanade-tokenizer.
- Vocoder Base: Weights and codebase derived from AliasingFreeNeuralAudioSynthesis.
- Training Techniques: Phase 3 training objectives were heavily inspired by XCodec2 and Inworld TTS-1.
🖊️ Citation
1@misc{miocodec-25hz-44.1khz,
2 author = {Chihiro Arata},
3 title = {MioCodec: High-Fidelity Neural Audio Codec for Efficient Spoken Language Modeling},
4 year = {2026},
5 publisher = {Hugging Face},
6 journal = {Hugging Face repository},
7 howpublished = {\url{https://huggingface.co/Aratako/MioCodec-25Hz-44.1kHz}}
8}
