An oQ8 quantized MLX version of Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp with integrated Multi-Token Prediction (MTP) head for accelerated speculative decoding on Apple Silicon.

Quantized with oMLX oQ8 — the highest-fidelity mixed-quantization variant that preserves the most layers at near-bf16 quality, providing the best accuracy among oQ variants at the cost of higher memory usage.

Model Summary

Format: MLX safetensors, oQ8 quantized
Precision: Mixed (oQ8 smart quantization)
Parameters: ~9B total
Context length: 1,048,576 tokens
Architecture: Qwen3.5-style hybrid attention text model with MTP head
MTP layers: 1 (embedded, mtp_num_hidden_layers: 1)
Primary use: local text generation and reasoning on Apple Silicon with MTP-accelerated decoding
Base model: xunkutech-ai/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp

Compatibility Notice

oMLX is required to load this model with MTP support. Standard mlx-vlm, mlx-lm, and LM Studio do NOT support MTP and will fail with: ValueError: Received 15 parameters not in model: language_model.mtp.*.

Installation

pip install -U git+https://github.com/jundot/omlx.git

Usage with oMLX

bash

# Start server with MTP draft-mode
omlx serve --model /path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ8-mtp

python

from omlx import load, generate

model_id = "/path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ8-mtp"
model, tokenizer = load(model_id)

result = generate(
    model=model,
    tokenizer=tokenizer,
    prompt="Explain speculative decoding with MTP.",
    max_tokens=2048,
    temperature=0.6,
)
print(result)

Files

File	Size	Description
`config.json`	~3.3 KB	oQ8 quantized config with `mtp_num_hidden_layers: 1`
`model-00001-of-00003.safetensors`	~5.0 GB	Quantized shard 1/3
`model-00002-of-00003.safetensors`	~5.0 GB	Quantized shard 2/3
`model-00003-of-00003.safetensors`	~0.7 GB	Quantized shard 3/3 (+ MTP head)
`tokenizer.json`	~20 MB	Qwen3.5 tokenizer
`chat_template.jinja`	~8 KB	Qwythos chat template

Recommended Sampling

python

generation_kwargs = {
    "temperature": 0.6,
    "top_p": 0.95,
    "top_k": 20,
    "repetition_penalty": 1.05,
    "max_tokens": 4096,
}

License

Released under the same license as the upstream model: Apache-2.0.

Acknowledgements

Original model: empero-ai/Qwythos-9B-Claude-Mythos-5-1M
Original developers: Empero AI
Base family: Qwen3.5
MTP support: oMLX
MLX ecosystem: Apple MLX

brainworkup/Qwythos-9B-v3-Claude-Mythos-5-1M-MLX-oQ8-mtp

Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ8-mtp