Quality-focused mixed-precision MLX/oMLX quantization with preserved native MTP and vision support for Apple Silicon.
In one sentence:KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX is a quality-oriented mixed-precision MLX/oMLX conversion of huihui-ai/Huihui-Qwen3.8-27B-abliterated, prepared as a lower-footprint local multimodal option with native MTP support on Apple Silicon.
This release is maintained by KostkaIT, an independent technology project by Łukasz Frąckowiak (vsnake87).
At a glance
Property
Value
Model family
Qwen3.8-27B
Upstream derivative
huihui-ai/Huihui-Qwen3.8-27B-abliterated
Release
oQ4e MLX quantization
Quantization
4-bit affine base, group_size: 64
Mixed-precision overrides
Selected tensors preserved at 5-bit and 6-bit
Format
MLX safetensors, 4 weight shards
Runtime target
oMLX and compatible MLX runtimes
Hardware target
Apple Silicon
Vision
Vision configuration, processor files and vision tensors included
Native MTP
Native MTP tensors included; one MTP hidden layer declared
MTP control
Enable or disable from compatible oMLX model Advanced settings
Primary use
Local coding, reasoning, multimodal assistants and research
Lower-footprint mixed-precision alternative with native MTP and vision assets.
Why choose oQ4e?
Choose this release when you want native MTP and multimodal capability in a more compact mixed-precision quantization.
4-bit affine base quantization with selected 5-bit and 6-bit tensor overrides.
Importance-matrix-assisted quantization rather than a uniform lowest-bit layout.
Native Qwen3.8 MTP weights preserved in the checkpoint.
Vision and processor assets included for multimodal workflows.
Native MTP can be controlled per model in compatible oMLX versions.
No separate external drafter is required for the native MTP path.
A practical alternative for users who want a lower-footprint option while retaining the same native-MTP model architecture.
If maximum quality preservation is more important than footprint, compare the oQ6e MLX variant.
Download — beginner-friendly
The following block is intended to be copied directly into Terminal. It installs the Hugging Face Hub CLI and downloads the complete model repository, including the model shards, tokenizer, chat template, configuration, processor files and safetensors index.
bash
1# Install or update the Hugging Face Hub CLI2python3 -m pip install -U huggingface_hub
34# Download the complete model repository5hf download KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX \6 --local-dir ./KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX
After downloading, keep the directory structure intact. Do not download only individual shard files: the index, tokenizer, processor and configuration files are required for a normal load.
Tip for beginners: use the copy icon in the upper-right corner of the code block, paste the complete block into Terminal, and press Enter.
The model is saved relative to the directory in which you run the command. To place it in a dedicated models folder, replace the destination with a path such as ~/Models/KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX.
Using with oMLX
Download the complete repository using the command above.
Add or select the downloaded model directory through the normal oMLX model-management workflow.
Open the model's Advanced settings.
Choose whether native MTP should be ON or OFF.
Reload the model if oMLX requests it.
Inspect the runtime log to confirm the active MTP path.
Manual editing of model_settings.json should not be required in a compatible oMLX version.
Do not enable a separate VLM-MTP or external drafter as a substitute for the native MTP weights included in this checkpoint.
Native MTP — what ON/OFF means
This repository contains actual native MTP checkpoint data under the language_model.mtp.* namespace. It is not only a configuration flag.
MTP ON: a compatible oMLX runtime may use the native MTP speculative decoding path. Confirm activation in the runtime log.
MTP OFF: the model can use the standard decoding path while keeping the same checkpoint and model files.
Reload: changing the setting may require a model reload before the new state is active.
Runtime boundary: successful model loading alone does not prove that MTP is executing; verify the runtime log.
Native MTP is different from:
a configuration flag without MTP tensors;
an external speculative-decoding drafter model;
the separate VLM-MTP path;
generic speculative decoding implemented by another runtime.
Vision — what is included
This is an image-text-to-text model package with vision assets included:
vision configuration;
processor configuration;
vision tensors;
image and video special tokens in the tokenizer;
the Qwen3.8 multimodal chat template.
The VISION badge means that the model repository contains the relevant multimodal assets. Actual image or video support still depends on the runtime and the workflow used to load the model. Text generation and native MTP loading do not automatically prove that every vision or video path works.
The upstream Huihui model is an abliterated derivative of Qwen3.8-27B. According to the upstream model card, the first 15 layers were retained without abliteration, while the MTP and visual components were not modified in that release.
The source revision used for this conversion was:
d42ca8978c5a66e92c3446d46e8adfe03ef692ff
This repository contains a quantized format conversion. It is not presented as a new fine-tune or as an official continuation of Qwen or Huihui.
Thanks to huihui-ai for creating and releasing the upstream model.
Quantization approach
This release uses oQ4e quantization created with an importance matrix (imatrix). The exact tensor layout and quantization metadata included in this repository are authoritative.
The mixed-precision layout was designed to prioritize:
preservation of model quality;
stability of the original behavior;
retention of sensitive model information;
native MTP compatibility;
predictable operation in oMLX.
The target was not the smallest possible model or the highest possible raw decode speed. Quantization can change the output distribution compared with the original BF16 model, so this release should not be considered bit-identical or lossless.
The local imatrix calibration cache and generation report are intentionally not included. They are not required for inference and may contain machine-specific paths and private calibration metadata.
Chat template and reasoning
Qwen3.8 uses a structured chat template with support for reasoning controls. Final behavior depends on the runtime and request parameters, including:
thinking enabled or disabled;
reasoning effort;
preservation of previous thinking content;
tool calling;
structured output;
context length;
sampling configuration.
Use the tokenizer and chat template shipped with this release. Replacing them with a generic template may change the model's behavior.
Runtime target and compatibility
This release primarily targets:
oMLX on Apple Silicon;
local or single-user inference;
coding and reasoning workloads;
multimodal experiments;
research and experimentation;
users who want per-model native MTP control without manual configuration edits.
Other MLX-compatible runtimes may load the model files, but native MTP support depends on the runtime. Always verify the MTP and vision paths separately in the runtime you intend to use.
Safety and responsible use
This is an abliterated model derivative. Abliteration changes refusal behavior; it does not make the model more truthful, more reliable or inherently safe.
Compared with a safety-aligned model, this model may produce content that the original model would refuse. It can also generate factually incorrect information, unsafe or harmful instructions, biased or inappropriate content, insecure code, and confident medical, legal or financial advice.
Users and deployers are responsible for applying safeguards appropriate to their use case.
Do not use this model as the sole decision-maker for:
medical or dietary treatment;
legal or financial decisions;
employment, housing, credit, education or insurance decisions;
autonomous actions affecting people or external systems;
content moderation without an additional moderation and review layer.
For public or multi-user deployment, implement authentication, authorization, input and output moderation, rate limiting, prompt and data isolation, logging, and human review for high-impact outputs.
Local inference can reduce network exposure, but it does not guarantee privacy. Operating-system services, runtimes, applications, logs, caches or monitoring tools may still retain prompts and generated outputs.
Limitations
Quantization may change model quality and output behavior.
Native MTP may be unavailable in unsupported runtimes.
Vision and video support must be tested separately.
Long-context behavior depends on memory and runtime configuration.
Tool calling and structured output require dedicated validation.
Results from one Apple Silicon device should not be generalized to all devices.
MTP is an inference optimization, not a safety mechanism.
The model can hallucinate and must not be treated as a source of truth.
FAQ
Is this a new Qwen or Huihui fine-tune?
No. This is an independent MLX quantized format conversion of the Huihui model. It is not an official Qwen or Huihui release.
Does native MTP require a separate drafter model?
No separate external drafter is required for the native MTP path when the target runtime supports the included MTP checkpoint data. Use the model-level MTP control in compatible oMLX versions and verify the runtime log.
Can native MTP be disabled?
Yes. Compatible oMLX versions expose MTP as a per-model Advanced setting. Set it to ON or OFF, reload if requested, and verify the selected state in the runtime log.
Does the VISION badge guarantee video support in every application?
No. The repository includes multimodal configuration and assets, but image and video support depends on the runtime and the specific loading path.
Should I choose oQ4e or oQ6e?
Choose oQ4e when a lower-footprint mixed-precision option is more convenient. Choose oQ6e when quality-oriented 6-bit quantization and higher preservation priority matter most.
License and attribution
The upstream Qwen3.8 and Huihui model repositories are marked as Apache-2.0. The full license text is included in LICENSE.
