Kokoro-82M Fast CPU GGUF
Project Overview
This is an optimized version of the Kokoro-82M TTS text encoder, designed specifically for high-performance CPU inference.
Key Optimizations
- Architecture: Unrolled the 12-shared-layer ALBERT encoder into 6 independent layers.
- Speed: Reduced depth by 50% for approximately 2x faster execution on CPU.
- Format: Converted to GGUF for seamless integration with C++ runtimes and efficient memory mapping.
- Weights: Pre-projected the 128-dim embeddings into a 768-dim space to eliminate runtime computation bottlenecks.
File Details
kokoro-fast-cpu.gguf: The optimized model binary.
Usage
Target runtime should support ALBERT-style GGUF models with 6 layers and a hidden dimension of 768.
