Qwen-3-TTS-12Hz-0.6B-Base-hi-LoRA-Finetuned/
├── README.md
├── config.json                  ┐
├── generation_config.json       │
├── merges.txt                   │
├── model.safetensors            ├─ bundled Qwen3-TTS-0.6B base model
├── preprocessor_config.json     │  (full precision — NOT quantized;
├── tokenizer_config.json        │   loads standalone, no separate download)
├── vocab.json                   │
└── speech_tokenizer/            ┘  12 Hz speech tokenizer (config + weights)
└── adapters/
    ├── hindi_female/
    │   ├── adapter_config.json
    │   ├── adapter_model.safetensors
    │   ├── config.json
    │   ├── ref.wav
    │   ├── ref.txt
    │   └── speaker_embedding.safetensors
    └── hindi_male/
        └── ...

The repo bundles the full, full-precision base model at root (plus speech_tokenizer/), so it loads standalone without separately downloading Qwen/Qwen3-TTS-12Hz-0.6B-Base. No quantization is applied to the base weights.

Each adapter folder contains:

adapter_config.json — LoRA configuration (rank 16, alpha 32)
adapter_model.safetensors — LoRA weight delta
config.json — base model config copy
ref.wav — reference audio clip for voice cloning (Hindi, that gender)
ref.txt — transcript of ref.wav (pass as ref_text)
speaker_embedding.safetensors — learned speaker embedding

Training Details

Setting	Value
Base model	`Qwen/Qwen3-TTS-12Hz-0.6B-Base`
Dataset	AI4Bharat Rasa — full Hindi split
Audio preprocessing	24 kHz, PCM-16, RMS-normalized to -20 dB
Clip duration	1-60 seconds
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Learning rate	1e-7
Epochs	2
Batch size	2 (gradient accumulation x4)
Precision	bf16
Attention	Flash Attention 2

Quickstart

bash

pip install -U qwen-tts peft transformers torch torchaudio
pip install -U flash-attn --no-build-isolation  # optional, for speed

python

import torch
import soundfile as sf
from huggingface_hub import snapshot_download, hf_hub_download
from qwen_tts import Qwen3TTSModel
from peft import PeftModel

REPO_ID    = "aguken-ai/Qwen-3-TTS-12Hz-0.6B-Base-hi-LoRA-Finetuned"
COMBO      = "hindi_female"   # or "hindi_male"
LORA_SCALE = 0.35             # tune between 0.1-0.5

BASE_MODEL   = snapshot_download(REPO_ID, allow_patterns=[
    "config.json", "generation_config.json", "merges.txt", "model.safetensors",
    "preprocessor_config.json", "tokenizer_config.json", "vocab.json", "speech_tokenizer/*",
])
ADAPTER_PATH = snapshot_download(REPO_ID, allow_patterns=[f"adapters/{COMBO}/*"]) + f"/adapters/{COMBO}"

model = Qwen3TTSModel.from_pretrained(
    BASE_MODEL, device_map="cuda:0", dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)
model = PeftModel.from_pretrained(model, ADAPTER_PATH)
model.set_adapter_scale(LORA_SCALE)

ref_audio = hf_hub_download(REPO_ID, filename=f"adapters/{COMBO}/ref.wav")
ref_text  = open(hf_hub_download(REPO_ID, filename=f"adapters/{COMBO}/ref.txt"), encoding="utf-8").read().strip()

text = "नमस्ते! मैं Avni बोल रही हूं Synoris Technologies से।"
wavs, sr = model.generate_voice_clone(text=text, language="Hindi", ref_audio=ref_audio, ref_text=ref_text)
sf.write("output.wav", wavs[0], sr)

LoRA Scale Tuning

Scale	Effect
0.1-0.2	Subtle Hindi influence, close to base model
0.3-0.4	Balanced — recommended starting point
0.5	Strong Hindi adaptation

Speaker IDs

Each adapter was trained with a dedicated speaker_id (rasa_hindi_female, rasa_hindi_male), stored in speaker_embedding.safetensors.

Dataset

Training data is the full Hindi split of AI4Bharat Rasa. Short clips are concatenated into ~60 s samples (audio + transcripts joined), filtered for:

Clip duration: 1-60 seconds
Clean text (no URLs, low digit ratio)
Audio normalized to -20 dB RMS at 24 kHz

License

Apache 2.0 — same as the base model.