You can deploy this model and start earning money today!

aidanakarib/
s2s-tur2kaz

Name: s2s-tur2kaz
Availability: InStock

speech-to-speech-translation

Views

No views yet

Direct Speech-to-Speech Translation: kaz → {tur, tat, uzb}

Trained checkpoints for a direct (no intermediate text at inference time) speech-to-speech translation system, Kazakh → Turkish / Tatar / Uzbek. Code: github.com/NLP-KazNU/s2s-public. Paper: IEEE Access, in revision (citation to be added on acceptance).

Architecture and checkpoints

File	Stage	Size
`whisper_s2tt_ws2tt_lv3.pt`	S2TT text stage: fine-tuned Whisper large-v3 encoder + custom FiLM decoder (`whisper_s2tt.py`)	2.7 GB
`codec_decoder_l9_head.pt`	Single-stream S2UT unit predictor: `CodecDecoderXAR`, n_q=1, mHuBERT-147 layer-9 units (`codec_decoder_xattn.py`)	122 MB
`unit_vocoder_hifigan.pt`	Dedicated Turkic single-stream unit vocoder (`train_unit_voc_hifigan.py`)	338 MB

Loading code and configs live in the GitHub repo; these .pt files are raw state_dict checkpoints matching the model classes defined there (WhisperS2TT, CodecDecoderXAR, SpeechBrain UnitHifiganGenerator).

Results (test set)

S2TT text-BLEU avg 19.05 (tur 12.76 / tat 25.76 / uzb 18.63)
End-to-end ASR-BLEU (argmax) avg 3.25 (tur 1.30 / tat 3.22 / uzb 5.24)
Beats an MMS→MADLAD→MMS cascade baseline (1.56 avg) by >2×

Full numbers, ablations, and the honest metric pack (chrF++/WER/CER/COMET/SQUIM/RTF) are in RESULTS.md.

Audio samples

See the demo_audio/ folder — 8 example system outputs (kaz source → predicted tur/tat/uzb speech) from the final single-stream S2UT system.

What this is not

Earlier architecture iterations (from-scratch encoder with morphology/VQ/SONAR auxiliary losses, a non-autoregressive mel decoder, an EnCodec-based acoustic-code predictor) all produced ASR-BLEU ≈ 0 and are not published — see the paper's Limitations section and the GitHub repo's README for why.

License

MIT.

aidanakarib/s2s-tur2kaz