You can deploy this model and start earning money today!

akashicmarga/
qwen3-tts-hindi-lora-grpo

Name: qwen3-tts-hindi-lora-grpo
Availability: InStock

mlx-community/Qwen3-TTS-12Hz-0.6B-Base-8bit

adapter

apache-2.0

Views

No views yet

Qwen3-TTS Hindi LoRA — GRPO-refined (Pipeline 3)

A Hindi LoRA adapter for Qwen3-TTS-0.6B, post-trained with GRPO (reinforcement learning) on top of the supervised akashicmarga/qwen3-tts-hindi-lora (SFT) adapter.

SFT optimizes next-token cross-entropy, which never directly targets intelligibility. GRPO optimizes it directly: the model samples speech, an ASR (whisper-large-v3-turbo) transcribes it, and CER vs the target text becomes the reward — so the policy is nudged toward audio a listener (proxied by ASR) can actually understand, while a KL penalty keeps it close to the SFT model.

Method


Algorithm	GRPO (group-relative, critic-free), strictly on-policy
Reward	intelligibility `1 − min(1, CER)` (whisper-large-v3-turbo) + length/degeneracy guard
Rollout layout	interleaved — matches the real `generate()` path (essential; the SFT-concatenated variant did not transfer)
KL reference	frozen SFT adapter snapshot, β = 0.08 (k3 estimator)
Prompts	863 Hindi "Read"-scenario sentences (IndicVoices-R)
Steps	400 (plateaued ~step 200), LR 5e-6, group size 4
Trainable	rank-8 LoRA on talker q/k/v/o/gate/up/down (462 tensors)

Evaluation (120 held-out sentences, 2 seeds, CER capped at 1.0)

	CER ↓	DNSMOS OVRL ↑	SIG ↑
SFT baseline	0.205	3.237	3.573
GRPO (this model)	0.183	3.271	3.608

CER −11% relative, better on 75/120 sentences (Wilcoxon p ≈ 7e-4; paired t p ≈ 0.09).
Naturalness (DNSMOS) small but significant gain (OVRL/SIG, p < 0.05) — no quality trade-off for the intelligibility gain.

A modest but statistically robust improvement. (DNSMOS is English-trained, so a rough relative proxy for Hindi naturalness; CER is an ASR proxy, not human MOS. GRPO reduces typical-case errors but does not eliminate occasional non-terminating hallucinations.)

Samples — SFT vs GRPO

Same sentence, same seed — only the adapter differs. These are normal-length held-out commands (no length gimmick); lower CER = more intelligible. Press play and compare.

Prompt (Hindi)	SFT baseline	GRPO (this model)	CER SFT→GRPO
फ़्रेशो सिग्नेचर हैंडमेड स्पिनच लवाश की कीमत एम आर पी से अधिक क्यों			1.63 → 0.18
फ़ोन नंबर … के लिए टीकाकरण प्रमाणपत्र डाउनलोड करने के लिए ओ टी पी …			1.03 → 0.45
मेरा ट्विटर नोटिफिकेशन चेक करें			0.61 → 0.06
मेरी सूची में जूते जोड़ो			0.42 → 0.06
इस विशिष्ट सूची में क्या है			0.26 → 0.00

(CER measured with whisper-large-v3-turbo; values >1.0 mean SFT produced more errors than characters, i.e. largely unintelligible on that prompt.)

Usage (MLX)

python

from mlx_audio.tts.utils import load_model
from train.lora import apply_lora, load_adapters, LoRAConfig

model = load_model("mlx-community/Qwen3-TTS-12Hz-0.6B-Base-8bit")
apply_lora(model, LoRAConfig(model_type="qwen3_tts", rank=8))
load_adapters(model, "adapters.safetensors")

for r in model.generate(text="नमस्ते, आज मौसम बहुत अच्छा है।", lang_code="auto"):
    pass  # r.audio is the waveform (24 kHz)

Trained on Apple Silicon (MLX). lang_code="auto" (nothink prefix — no dedicated Hindi token).

License

Apache-2.0. Built on IndicVoices-R (CC-BY-4.0).

akashicmarga/qwen3-tts-hindi-lora-grpo