Qwen3.6 NVIDIA VoiceChat Stage A adapters
These are experimental Stage A adapters for replacing the semantic backbone of
nvidia/NVIDIA-NemotronLabs-VoiceChat-11B with Qwen/Qwen3.6-35B-A3B while preserving
NVIDIA's FastConformer, AddFusion, output/timing heads, RNNT, DuplexEARTTS talker, and codec.
This checkpoint is not a working speech-to-speech model. It passes acoustic-dependence
controls at its parent content-margin stage, but the latest unseen free-running diagnostic
fails semantic coherence and repeats meta-text. It is published for reproducibility and
continued research, not deployment.
Files
perception_projection.safetensors: trained NVIDIA perception projection adapter.qwen_lora.safetensors: rank-8 LoRA weights for the upper Qwen layers.nvidia_voicechat_qwen36_120ms.json: exact composite graph configuration.checkpoint_metadata.json: dependencies, hashes, training objective, and gate result.failed_preview/: the held-out input and generated text proving the current failure.
The repository intentionally does not redistribute either required base checkpoint.
Required bases
nvidia/NVIDIA-NemotronLabs-VoiceChat-11B, revision
5631f538c74d1b4a8adfbc0b3a2c4aed6eba4d56Qwen/Qwen3.6-35B-A3B, BF16 revision
995ad96eacd98c81ed38be0c5b274b04031597b0- NVIDIA Speech branch
nemotron-labs-voicechat, commit
097dfe9e2f55baf653b83035868bdc89849f1b47
Base-model licenses and access terms apply. The NVIDIA checkpoint uses OpenMDW 1.1; consult
the original model cards before use.
Training state
The adapter was trained on real SpokenWOZ conversational pairs with the SALM-Duplex paper
clock/channel curriculum, offline Qwen top-64 text-path distillation, ground-truth response
CE, compact user-end representation alignment, text replay, scheduled sampling, and a
duration-aligned wrong-audio margin. Training and inference use direct Hugging Face
Transformers/PyTorch; vLLM is not used.
The operational fallback frame length is 120 ms because the strict 80 ms direct-HF serving
gate did not pass on the test GH200.
Gate result
FAIL_M2A_FREE_RUNNING_GATE
For the unseen user audio “Hello, is this Customer Service Center?”, this checkpoint emitted
repeated “I will provide a response in English.” No talker/codec output was accepted. Continue
with encoder co-adaptation or scheduled continuous-modality mixing before attempting later
duplex-policy stages.
