Qwythos-9B-Claude-Mythos-5-1M — MLX MXFP4
MXFP4 (4-bit) MLX conversion of
empero-ai/Qwythos-9B-Claude-Mythos-5-1M, for running on Apple Silicon with
MLX /
mlx-vlm.
Details
| |
|---|
| Base model | empero-ai/Qwythos-9B-Claude-Mythos-5-1M (Qwen3.5-9B fine-tune) |
| Architecture | qwen3_5 (multimodal) — hybrid Gated DeltaNet linear + full attention |
| Quantization | MXFP4, group size 32, ~4.82 bits/weight |
| Size | ~5.3 GB (from 18.8 GB BF16) |
| Vision | Preserved — vision tower + processors kept; runs image/video + text |
| Context | up to 1M tokens via YaRN (inherited from base) |
| License | Apache-2.0 |
Note: in the base model only the text path was fine-tuned (on "Claude Mythos/Fable" traces). The vision tower is inherited unchanged from Qwen3.5-9B base, so visual quality matches the base model.
Benchmarks
Measured on Apple M4 · 16 GB unified memory, MLX 0.31.2 / mlx-vlm 0.6.3, greedy decoding (temperature = 0), text-only path.
| Metric | Value |
|---|
| Precision | MXFP4 (4-bit), group size 32, ~4.82 bits/weight |
| On-disk size | 5.3 GB (from 18.8 GB BF16 → ~3.5× smaller) |
| Load time | 3.5 s |
| Peak memory | 6.1 GB (short context) → 7.5 GB (~5.7k tokens) |
| Generation speed | ~20.2 tok/s (256-token output) |
| Prompt processing | ~127 tok/s @ 122-tok prompt · ~157 tok/s @ 5.7k-tok prompt |
| Perplexity (EN sample) | 6.73 |
Quality spot-checks (greedy): "capital of Japan" → Tóquio ✓ · 17 × 23 → 391 ✓.
Speeds are single-stream on a base M4; Pro/Max chips and larger memory bandwidth will be faster. The model runs comfortably within 16 GB RAM with room for context.
Evaluation
On a 45-item Portuguese eval covering factual knowledge, arithmetic, logic/reasoning, strict instruction-following, and code (executed against unit tests), auto-graded with greedy decoding (temp = 0), this MXFP4 build scored 97.8 % overall:
| Category (n) | Score |
|---|
| Factual (15) | 100 % |
| Math (10) | 100 % |
| Logic / reasoning (8) | 87.5 % |
| Instruction-following (6) | 100 % |
| Code — executed (6) | 100 % |
In our testing the model held up well against other 4-bit Qwen3.5-9B community builds, at comparable speed (~21 tok/s) and memory (~6 GB peak on an Apple M4). The MXFP4 quantization preserves quality well — the only miss was a classic counterintuitive "rate" word puzzle.
Caveat: 45 items, single greedy run — a useful signal, not a formal large-scale benchmark (e.g. MMLU). Vision was not evaluated.
Usage (mlx-vlm)
1pip install -U mlx-vlm
2python -m mlx_vlm generate \
3 --model <THIS_REPO> \
4 --prompt "Describe this image." \
5 --image path/to/image.jpg \
6 --max-tokens 256
Text-only also works (omit --image).
Conversion
1python -m mlx_vlm convert \
2 --hf-path empero-ai/Qwythos-9B-Claude-Mythos-5-1M \
3 --mlx-path ./Qwythos-9B-Claude-Mythos-5-1M-MLX-MXFP4 \
4 -q --q-mode mxfp4 --q-bits 4 --q-group-size 32
Quantized with
mlx-vlm on Apple Silicon. All credit for the base model goes to
empero-ai.
