Views
No views yet
Not a full LLM. Load only as--draft-modelnext to an MLX VLM target.
| Repo | Role |
|---|---|
| VLM 4-bit | Main model (recommended) |
| VLM BF16 | Main model (high RAM) |
| This MTP drafter | Optional decode speed-up |
mlx_vlm.convert drops mtp.*. This repo puts them back as model_type: qwen3_5_mtp (block_size=4, ~829MB).pip install -U mlx-vlm1mlx_vlm.generate \
2 --model FancieF/Qwythos-27B-v1-MLX-VLM-4bit \
3 --draft-model FancieF/Qwythos-27B-v1-MLX-MTP \
4 --draft-kind mtp \
5 --draft-block-size 4 \
6 --prompt "Explain gated linear attention in three sentences." \
7 --thinking-mode disabled \
8 --temperature 0.61mlx_vlm.generate \
2 --model FancieF/Qwythos-27B-v1-MLX-VLM-4bit \
3 --draft-model FancieF/Qwythos-27B-v1-MLX-MTP \
4 --draft-kind mtp \
5 --draft-block-size 4 \
6 --image ./photo.jpg \
7 --prompt "Describe this image." \
8 --thinking-mode disabled1mlx_vlm.server \
2 --model FancieF/Qwythos-27B-v1-MLX-VLM-4bit \
3 --draft-model FancieF/Qwythos-27B-v1-MLX-MTP \
4 --draft-kind mtp \
5 --draft-block-size 4*MTP*Q4_K_M*) + enable MTP Speculative Decoding.1python -m mlx_vlm.speculative.drafters.qwen3_5_mtp.split \
2 --model /path/to/Qwythos-27B-v1 \
3 --output ./Qwythos-27B-MTP \
4 --block-size 4| Main VLM 4-bit | FancieF/Qwythos-27B-v1-MLX-VLM-4bit |
| VLM BF16 | FancieF/Qwythos-27B-v1-MLX-VLM-bf16 |
| Base | empero-ai/Qwythos-27B-v1 |
| ModelScope (ZH) | Fancie/Qwythos-27B-v1-MLX-MTP |