RumiLabs voice-AI stack — STT + emotion sensing + 4B reasoning LLM, sharing one frozen audio encoder. Built for iPhone-class on-device deployment. Beats whisper-small on LibriSpeech test_clean (WER 0.0269 vs 0.0339) at ~40× smaller disk than whisper-large-v3-turbo for the STT path; with decode-stack extensions (chapter-context conditioning + margin-gated beam-200 escalation, zero extra ship size) reaches 0.0254 — within 0.0038 of turbo (0.0216 greedy decode, matched normalizer).

This is the release repo. For the staging bundle with intermediate STT checkpoints (tiny / small / turbo + Phase-by-phase snapshots), see RumiLabs/MOSS-Audio-4B-Thinking-2507-MLX-4bit-v2. For the BF16 source weights see RumiLabs/rumi-cerberus-4b-bf16.

LibriSpeech (full sets, whisper.normalizers.EnglishTextNormalizer):

¹ Reference implementation in the repo (audio/scripts/rescore_campaign/): condition the rescorer on the previous two predicted transcripts (streaming-legal) + style prompt; escalate the 30% lowest-margin utterances to a beam-200 pool. Dev-tuned, single test evaluation. Swift port shipped in swift/LLMRescorer.swift (contextPrefix + margin-gated escalation; env-gated in the Yobi harness) — validated on iPhone 16 Pro Max 2026-06-03: pick-parity with the reference recipe on the English fixture clips, plus a live-microphone end-to-end pass (capture → conditioned rescore → gated escalation, memory-stable).

Which row to use: the two rows are different operating points, not strict dominance. rescore (0.0269) is the simple, flat-latency (~2.5 s/utt), fully device-validated default — stateless per utterance, emits empty on silence/noise. + decode extensions (0.0254) is the max-quality mode for dictation/transcription flows: same p50 latency but the ~30% lowest-margin utterances escalate (+5–8 s, p95 worse); the conditioning style prompt can bias toward emitting text on garbled/noisy input (silence + word-rate guards included); part of the gain is corpus-flavored (sequential-context + style prompt — the conversational-domain delta is unquantified); and it conditions on the previous two transcripts, so a bad prior turn can propagate (capped at 2). ² Baseline verification 2026-06-02: all references re-measured on the identical clips under the same EnglishTextNormalizer scoring as our numbers (an earlier card cited turbo 0.0285 from a strict-normalizer evaluation; the shipped recipe beat that ship-time reference, and the decode extensions close most of the re-measured gap).

The rescore stage reuses the same Qwen3-4B already in the bundle for reasoning / tool-calling — zero additional ship size. Recipe:

Sum-not-mean LLM scoring + beam-200 + whisper text normalizer. See mlx_stt/README.md for full inference recipe.

The Qwen3-4B-Thinking-2507 LLM in mlx_llm/ is jointly trained with the frozen audio encoder via Stage 2.1 LoRA — supports audio-conditioned chat, tool-calling, and reasoning. Audio benchmark: 7/7 fixture clips pass. BFCL v3 (3-cat avg): 91.17%.

iPhone 16 Pro Max: 17.85 tokens/s decode, 0.51 s TTFT for the LLM path. All three heads (STT, emotion, LLM) share one audio-encoder forward pass.

The full speech-to-speech turn (mic → encoder → STT+emotion → LLM reasoning → streamed Qwen3-TTS reply) ships as a validated configuration. Reference Swift sources under swift/tts/ (integration layer: TTSSpeaker.swift, QwenTTSSpeaker.swift — streaming orchestration, restore-during-playback, closed-loop prebuffer).

Measured on iPhone 16 Pro Max (Release ≡ debug — compute is Metal kernels):

Robustness guards (in the integration layer): MLX cache capping for

Emotion-styled replies: the affect head's label selects the TTS instruct (angry→calm, happy/excited→cheerful). The "sad" label is demoted to a neutral voice by default — the head over-reports sad on natural conversational mic speech (acted-corpus domain gap); YOBI_TTS_TRUST_SAD=1 restores full trust.

Env knobs: YOBI_QWEN_STREAM=0 (whole-reply synthesis), YOBI_QWEN_STREAM_BUFFER (fixed prebuffer), YOBI_ACK=0 (no acknowledgment clips), YOBI_TTS_EMOTION_STYLE=0 (fixed neutral voice), YOBI_TTS=avspeech (system TTS fallback).

Private repo — requires RumiLabs org access.

If you use this in research, please cite the upstream models (MOSS-Audio, Qwen3) and our forthcoming workshop paper (NeurIPS 2026 Workshop submission, in preparation).