An OpenVINO INT4 (weight-only) build of the text decoder extracted from empero-ai/Qwythos-9B-Claude-Mythos-5-1M, a qwen3_5 "omni" (image/audio/video) model based on Qwen/Qwen3.5-9B.

The source is a tri-modal VL model. This derivative keeps only the language_model + lm_head, grafted into a standalone text-config CausalLM, then exported to OpenVINO IR and quantized to INT4 (symmetric, weight-only, group size 128, ratio 1.0) via optimum-intel / OpenVINO / NNCF. Text-only — no vision/audio.

Loads and generates under OpenVINO Model Server on Intel Arc (GPU), ~80 tok/s at INT4 (~5.6 GB weights).

On OVMS builds bundling openvino-genai 2026.2.0, the omni tokenizer's chat template fails to load in the serving path (/v3/chat/completions → "Chat template not loaded correctly"). /v3/completions (raw prompt) works. The fix (chat_template_mode: MINJA, or genai ≥ 2026.2.1) is upstream; until a newer OVMS ships, front /chat with a small templating proxy that applies the tokenizer's chat_template via transformers and forwards to OVMS.

This is a reasoning model that emits extended chain-of-thought; for latency-sensitive or strict-JSON tasks, use grammar-constrained decoding to keep output concise.