Nex-N2-mini-MLX-VLM-8bit-MTP
Native MLX-VLM 8-bit quantized version of nex-agi/Nex-N2-mini, with a grafted MTP head for oMLX Native MTP speculative decoding.
Summary
- Base model:
nex-agi/Nex-N2-mini
- Format: native MLX / MLX-VLM
- Quantization: 8-bit MLX-VLM quantization
- Vision: supported
- MTP: included
- Target runtime: oMLX with Native MTP enabled
- Direct
mlx-vlm.generate: not the supported runtime for this MTP variant
What changed in this version
This repository uses the native MLX-VLM trunk and vision weights from Nex-N2-mini-MLX-VLM-8bit, while the MTP head is grafted from the jedisct1/Nex-N2-mini-mlx-OptiQ-8bit-MTP MTP variant.
This is not the same as jedisct1/Nex-N2-mini-mlx-OptiQ-8bit-MTP:
- This repo keeps the native MLX-VLM layout.
- This repo includes the vision tower / VLM weights.
- The
jedisct1 OptiQ MTP repository is text-only.
- The MTP head is used only for speculative decoding acceleration.
Important compatibility note
This MTP variant includes language_model.mtp.* weights intended for oMLX Native MTP.
The validated runtime is:
- oMLX
- Native MTP enabled in model settings
Plain mlx-vlm.generate or generic MLX loading paths may fail strict loading or may not use the MTP head correctly, because the preserved language_model.mtp.* tensors are intended for the oMLX Native MTP runtime.
For direct mlx-vlm.generate usage, use the non-MTP variant instead:
joowon-jang/Nex-N2-mini-MLX-VLM-8bit
Quality and behavior
The MTP head is used for speculative decoding. Draft tokens are verified by the main Nex-N2-mini trunk before being accepted, so the MTP head is intended to affect speed rather than final output quality.
In practice, MTP speedups depend on draft acceptance rate. The grafted head tends to help more on normal prose and reasoning, and less on unusual token sequences.
Recommended runtime
Use oMLX and enable Native MTP.
Notes
This is not an OptiQ oQ8 sidecar model. The model uses a native MLX-VLM layout with vision_tower.* weights included in the model files.
MTP head attribution:
- MTP head source:
jedisct1/Nex-N2-mini-mlx-OptiQ-8bit-MTP
- Original base model:
nex-agi/Nex-N2-mini
- Donor MTP lineage described by the source model: Qwen3.5-35B-A3B MTP head grafted onto Nex-N2-mini-compatible dimensions
License
Apache-2.0, following the base model and referenced MTP-head source license.
