Qwopus3.5-4B-Coder-MTP-oQ3.5
An oMLX oQ-quantized version of Qwopus3.5-4B-Coder-MTP optimized for efficient local inference on Apple Silicon devices.
About Qwopus3.5-4B-Coder
Qwopus3.5-4B-Coder is a compact coding and agent-oriented model built on the Qwen3.5 4B family.
The model is designed for:
- Coding assistance
- Agent workflows
- Tool use
- Debugging
- Structured reasoning
- Software engineering tasks
- Local development environments
The training recipe combines reasoning-oriented techniques, agent trajectories, and coding-focused instruction tuning to improve stability and practical coding performance.
About This Quantization
This repository contains an oMLX oQ3.5 mixed-precision quantization of the original model.
Unlike traditional uniform quantization methods, oQ allocates precision dynamically according to layer sensitivity. Critical model components retain higher precision while less sensitive components are compressed more aggressively.
Benefits include:
- Reduced memory consumption
- Reduced storage requirements
- Better quality retention than uniform low-bit quantization
- Faster local inference
- Improved efficiency on Apple Silicon hardware
Multi-Token Prediction (MTP)
This release preserves the model's Multi-Token Prediction (MTP) components.
MTP allows the model architecture to predict multiple future tokens internally, improving generation efficiency and helping maintain compatibility with runtimes and workflows that support MTP-enabled Qwen-family models.
Recommended Settings
For best results:
1temp: 1.0
2top_p: 0.95
3top_k: 20
4min_p: 0
5rep_penalty: 1.05
6presence_penalty: 1.5
7enable_thinking: true
These settings provide a good balance between exploration, acceptance rate, and generation quality when paired with a Qwen3.5 target model.
Consider using DFlash model for more accurate and faster response.
https://huggingface.co/z-lab/Qwen3.5-4B-DFlash or
https://huggingface.co/yugeshkarunamurthy/Qwen3.5-4b-Dflash-6bit-MLXIntended Use
This model is suitable for:
- Code generation
- Code review
- Debugging assistance
- Agentic coding workflows
- Terminal assistants
- IDE integrations
- Research and experimentation
- Local AI development
Usage
MLX-LM
1from mlx_lm import load, generate
2
3model, tokenizer = load("path/to/model")
4
5response = generate(
6 model,
7 tokenizer,
8 prompt="Write a Python function that implements binary search.",
9 max_tokens=512,
10)
11
12print(response)
Claude Code
This model works well as a local coding model for Claude Code workflows where fast iteration, code generation, debugging, and repository assistance are required.
Quantization Details
| Item | Value |
|---|
| Base Model | Qwopus3.5-4B-Coder-MTP |
| Quantization Method | oMLX oQ3.5 |
| Format | MLX |
| MTP Preserved | Yes |
| Architecture | Qwen3.5 Family |
Performance Notes
Performance depends on:
- Context length
- Runtime implementation
- Hardware configuration
- Quantization parameters
- Prompt style
Users are encouraged to benchmark the model on their own workloads.
Limitations
This model inherits the strengths and limitations of the original Qwopus3.5-4B-Coder model.
Quantization may introduce:
- Minor reductions in reasoning quality
- Small changes in generation behavior
- Reduced performance on certain edge-case tasks
Results will vary depending on hardware and inference settings.
Credits
Original Model
- Jackrong — Qwopus3.5-4B-Coder-MTP
Quantization
Citation
If you use the original model in research, please cite the original Qwopus model authors and repository.
Disclaimer
This repository contains a community-generated quantized checkpoint and is not an official release from the original model authors.
Please evaluate the model carefully before deploying it in production environments.
