You can deploy this model and start earning money today!

marcosremar2/
ptbr-omni-qwen3-venus-iasini-v1

Name: ptbr-omni-qwen3-venus-iasini-v1
Availability: InStock

Views

No views yet

PT-BR Omni Qwen3/Tucano Prototype - venus_iasini v1

Private research snapshot for the PT-BR omni voice pipeline:

audio question -> Whisper tiny -> audio bridge -> Tucano 2 -> answer text -> speech output

This repository preserves the useful training and benchmark artifacts from the GPU run. It does not include raw YouTube/reference audio. It includes generated validation WAVs, training manifests, checkpoints, and the exact Python tools used for the experiments.

What Works

The audio bridge understands the Portuguese audio question and feeds Tucano coherently.
The Kokoro fast answer baseline is intelligible and reaches sub-1s first playable audio after warmup.
Qwen3 decoder/token layout is validated with ground-truth Qwen3 codes: target codes decode to intelligible PT-BR audio.

Current Limitation

The small Tucano -> Qwen3 talker is not quality-approved yet. It can produce a first chunk in about 0.8s, but the full generated audio is not intelligible. The next technical step is improving the talker/code predictor, not replacing the Qwen3 decoder.

Key Results

Path	Warm protocol	Result
Kokoro fallback answer	3 warmups + 3 measured	median first playable audio 0.822s
Audio bridge + Tucano text	3 warmups + 3 measured	median to answer text 0.760s
Small Qwen3 talker first chunk	3 warmups + 3 measured	median first chunk 0.796s
Qwen3 teacher full answer	3 warmups + 3 measured	median first playable audio 7.141s
Qwen3 ground-truth code decode	3 probe rows	ASR similarity about 0.864-0.943

Important Checkpoints

checkpoints/tucano_qwen3_tts_venus_iasini_audio_bridge_aqa_kokoro_v1/audio_bridge.pt
- Whisper/audio-feature bridge trained on the PT-BR audio Q/A bootstrap curriculum.
checkpoints/tucano_qwen3_tts_venus_iasini_talker_aqa_answers_v1/talker.pt
- AR C0 Qwen3 talker fine-tune. Kept for reproducibility; not production-quality.
checkpoints/tucano_qwen3_tts_venus_iasini_talker_direct_aqa_answers_v1/talker.pt
- Direct Qwen3 talker fine-tune used in first-chunk experiments. Fast, but not intelligible enough.

Training Manifests

data/audio_qa/venus_iasini/manifests/venus_iasini_audio_qa_curriculum.jsonl
data/audio_qa/venus_iasini/manifests/venus_iasini_audio_qa_kokoro_bootstrap_train.jsonl
data/audio_qa/venus_iasini/manifests/venus_iasini_audio_qa_answers_qwen3_codes.jsonl

Each manifest has 96 examples. The Qwen3-code manifest stores answer text, speaker id, and flattened Qwen3 audio codes.

Benchmarks And Audio Probes

artifacts/benchmark_page/index.html
- Local benchmark dashboard copied from the run.
artifacts/qwen3_omni_venus_kokoro_fast_answer_bench/
- Fast intelligible fallback, voice pf_dora, not venus_iasini.
artifacts/qwen3_omni_venus_gt_code_decode_probe/
- Ground-truth Qwen3 token decode probes validating the decoder/token layout.
artifacts/qwen3_omni_venus_first_chunk_small_talker_bench/
- True first-chunk benchmark for the small Qwen3 talker.
artifacts/qwen3_omni_venus_small_talker_full_audio_probe/
- Full-audio probes showing the current small talker intelligibility failure.
artifacts/qwen3_omni_venus_audio_qa_gpu_functional_qwen_teacher/
- Functional Qwen3 teacher voice-clone baseline; quality OK, too slow for real-time UX.
artifacts/qwen3_omni_venus_audio_qa_warm_first_chunk_bench/
- Warm Qwen3 teacher benchmark.

Reproduction Notes

Expected base models:

Polygl0t/Tucano2-qwen-1.5B-Instruct
Qwen/Qwen3-TTS-Tokenizer-12Hz
faster-whisper tiny
Kokoro for the fast fallback baseline

Main scripts are under tools/talker_text_qwen3_tts/.

Status

This is a private experimental checkpoint. Treat the bridge and the benchmark harness as the working pieces. Treat the Qwen3 talker checkpoints as reproducibility artifacts that need another training iteration before mobile export.

marcosremar2/ptbr-omni-qwen3-venus-iasini-v1