So I want an LLM to teach me Japanese and I have an NVIDIA Thor. I have compressed models text layers to fit
into available memory. I also included patched llmcompessor, inference script and a proof of concept script that
does a multi turn voice conversation with vllm-omni in extras.
This model quantizes all layers which works Ok for voice recognition and text generation, but results in garbled voice. Keeping it because
it's useful with external TTS that can give more voice options. Use catplusplus/Qwen3-Omni-30B-A3B-Instruct-NVFP4-talker-safe for voice
generation, the tradeoff is larger size.
