LiteRT / TensorFlow Lite port of openbmb/VoxCPM2 — a 2B-parameter multilingual diffusion-autoregressive TTS model with 48 kHz studio-quality output, voice cloning, and instruction-driven voice design.

This bundle ships the model as four separate LiteRT graphs plus a manifest. The on-device worker is expected to orchestrate the loop:

text_prefill ─► token_step (×N) ─► audio_decode

Reference audio is encoded once via audio_encoder. The K/V cache for the LM and residual decoder is owned by the host worker (not mutated inside the graph), which lets the runtime own retry / idempotency semantics.

Part of soniqo.audio — an on-device speech toolkit. Consumed by the Android SDK at speech-android.

Status: experimental. The token_step graph depends on litert_torch static K/V-cache lowering; integrators should validate numerical parity end-to-end before relying on this bundle in production.

Capabilities

30 languages including English, Chinese, Indonesian, Japanese, Korean
48 kHz output
Zero-shot synthesis — generate speech from text alone
Voice cloning — clone a target speaker from a single reference clip
Voice design — natural-language style control (e.g. "young female voice, warm and gentle")
Ultimate cloning — reference audio + transcript for prosody-preserving cloning

Files

File	Variant	Role
`voxcpm2-text-prefill.tflite`	FP16 weights / FP32 compute	Encode text + (optional) reference-audio prefix into LM hidden states, residual hidden, prefix feature conditioning, and the initial K/V caches.
`voxcpm2-token-step.tflite`	FP16 weights / FP32 compute, LocDiT diffusion kept FP32	One AR step. Takes current LM / residual hidden, conditioning, K/V cache and position id. Emits the next predicted feature, stop logits, updated hidden states, and updated K/V cache.
`voxcpm2-audio-encoder.tflite`	FP32	Encode a reference clip (16 kHz PCM) into the patch features that condition the prefill.
`voxcpm2-audio-decoder.tflite`	FP32	Decode latent → 48 kHz PCM via the upstream AudioVAE.
`config.json`	—	Manifest: tensor signatures, sample rates, default CFG / step counts, file mapping.
`tokenizer.json` / `tokenizer_config.json` / `special_tokens_map.json` / `generation_config.json`	—	HF tokenizer + generation defaults.
`tokenization_voxcpm2.py`	—	Upstream tokenizer source (kept for parity with the HF model).

The encoder and decoder are kept FP32 (the AudioVAE is sensitive to quantisation). The LM graphs are FP16-weight / FP32-compute except the LocDiT diffusion estimator, which is held at FP32 — sub-FP32 precision there turns the noise-like texture of consonants into tonal structure, producing sandy/metallic sibilants on cloned voices. This selective recipe matches full-FP32 cloning quality at roughly half the size (~8.7 GB vs ~17 GB).

Default decoding parameters

Parameter	Default
`max_text_tokens` (context)	512
`max_generated_tokens`	2048
`inference_timesteps` (CFM)	10
`cfg_value`	2.0
Sample rate (output)	48 000 Hz
Sample rate (audio conditioning)	16 000 Hz

These mirror the host-side defaults exposed in config.json; runtimes are free to override them.

Token-step cache contract

The token-step graph takes and returns the LM and residual K/V cache as explicit inputs/outputs. Cache layout:

[2, layers, batch, kv_heads, max_cache_length, head_dim]

Axis 0: [K, V]
Axis 4 is sized to max_text_tokens + max_generated_tokens and pre-allocated by the worker.
The graph does not mutate the cache buffers in place — it produces updated tensors which the worker copies / swaps.

This contract is what makes parallel decoding, mid-generation cancellation, and deterministic replay possible from the C++ side.

Source

Converted from the upstream PyTorch weights at openbmb/VoxCPM2 using litert_torch + ai_edge_quantizer float-casting (FP16 weights, FP32 compute), with the LocDiT diffusion estimator held at FP32 — the selective recipe (convert_litert.py --variant selective).

License

Apache 2.0 (inherited from upstream openbmb/VoxCPM2).

Responsible use

Voice cloning capability is included. Users are responsible for obtaining consent for any voice that is cloned and for not using the model to impersonate individuals without their permission, generate disinformation, or commit fraud.

soniqo/VoxCPM2-LiteRT

VoxCPM2 LiteRT