A 8-bit MLX quantization of
trjxter/Qwimi-3.6-27B-Coder-MTP-BF16, a coding-focused SFT of Qwen 3.6 27B.
The model was trained in one mixed SFT run across coding, Qwen-native XML-style tool calling,
and agentic software-engineering trajectories. This repository is intended for local
inference on Apple silicon using mlx-lm.
This is an MLX repository, not GGUF. It is not intended for llama.cpp, CUDA inference,
or a GGUF-only Ollama workflow.
Quant position: highest-fidelity quantized MLX release. Primary tradeoff: smallest quantization loss of the Q4/Q6/Q8 set. Recommendation: Best when quality is the priority and the model fits comfortably.
The uploaded repository's config.json is the source of truth for quantization mode, group
size, and tensor metadata. This README intentionally does not invent a group size that may
differ from the actual conversion.
The source name includes MTP. Do not assume that conversion to MLX automatically preserves
or activates the same MTP runtime path used by a specific llama.cpp build.
2. Which MLX quant should I use?
Choose Q8 when you want the closest of these three quantized releases to BF16 behavior and
your Mac has enough unified memory.
Prefer Q6 for a better size/quality balance or Q4 when memory and context headroom dominate.
MLX release
Relative memory use
Expected fidelity
Best fit
Q4
Lowest
Good, with the most quantization loss
Constrained memory and more context headroom
Q6
Medium
Strong balance
General recommendation
Q8
Highest
Highest of the three
Maximum quality on larger-memory Macs
Exact quality and memory behavior should be measured with the uploaded artifacts and your own
workload.
3. Unified-memory guidance
Practical target: 64 GB unified memory or more.
48 GB Macs: may work with controlled context and limited competing memory use.
96 GB+ Macs: provide generous headroom for long prompts, cache, IDEs, and agents.
These are rough planning targets, not guarantees. Actual memory use depends on the exact file
size, group size, prompt length, generation length, cache configuration, macOS version, and
other applications using unified memory.
Qwen 3.6 / qwen3_5 support is relatively new, so update mlx-lm before diagnosing model,
cache, server, or tool-parser problems.
5. How to run
One-shot generation
bash
1mlx_lm.generate \2 --model trjxter/Qwimi-3.6-27B-Coder-MTP-MLX-8bit \3 --prompt "Write a Python function that reverses a linked list and include tests."
mlx_lm.server is most appropriate as a local development endpoint, not a hardened public
production service.
Context guidance
The base architecture may expose a larger native context, but Qwimi was trained and validated
at 16,384 tokens. Longer-context behavior should be tested separately.
6. Tool calling
Qwimi was trained on Qwen 3.6's native XML-style tool-call format rather than the older plain
JSON-object style.
Validate multi-turn tool calls, inserted tool results, and stop behavior.
Do not assume that working text generation guarantees correct server-side tool parsing.
Keep direct XML validation or a fallback parser for agent workflows.
7. Training summary
Base:unsloth/Qwen3.6-27B
Model size: approximately 27.8B parameters
Method: 4-bit QLoRA during SFT, rank 64, alpha 64
Trainable parameters: 466,911,232, approximately 1.68%
Training style: one mixed SFT run, not three sequential stages
Total training rows: 22,359
Training tokens: approximately 72.5M
Maximum sequence length: 16,384
Effective batch size: 16
Epochs: 1
Optimizer steps: 1,398
Hardware: 1× A100-SXM4-80GB
Vision tower: frozen
Final eval loss: coding 0.4761; tool-calling 0.0208; agentic 0.2892
All three eval curves improved monotonically through the run with no observed overfitting signal.
Domain
Train rows
Eval rows
Token share
Coding
16,083
1,778
82.5%
Tool-calling
5,360
595
4.8%
Agentic
917
102
12.7%
trjxter/Kimi-K2.7-CodingTraces-9000x was used in full: 9,014 rows.
8. Benchmarks
Qwimi was compared against the base Qwen 3.6 27B model using equivalent
Q6 GGUF builds with MTP enabled and the same evaluation harness.
Important: these results were produced by the Q6 GGUF / llama.cpp builds. They are
included as evidence about the underlying Qwimi model, not as direct benchmarks of this MLX
quant. MLX Q4, Q6, and Q8 should be benchmarked separately before making backend- or
precision-specific claims.
8.1 Fully auto-scored 300-task custom benchmark
Category
Tasks
Coding
150
Tool-calling
75
Agentic workflows
75
Total
300
Model
Tasks
Pass rate
Average wall time
Total wall time
Base Qwen 3.6 27B Q6 MTP
300
81.33%
53.56 s
267.79 min
Qwimi 3.6 27B Coder Q6 MTP
300
82.67%
27.09 s
135.45 min
Overall change
Result
Absolute pass-rate gain
+1.33 percentage points
Relative pass-rate gain
+1.64%
Average wall-time reduction
49.4%
Effective task-level speedup
~1.98×
Total evaluation time saved
132.34 minutes
Category results
Category
Base pass rate
Qwimi pass rate
Accuracy delta
Base avg time
Qwimi avg time
Wall-time reduction
Custom coding
81.33%
85.33%
+4.00 pts
59.33 s
21.16 s
64.3%
Custom tool-calling
88.00%
89.33%
+1.33 pts
31.77 s
27.13 s
14.6%
Custom agentic
74.67%
70.67%
-4.00 pts
63.80 s
38.92 s
39.0%
Pairwise outcomes
Pairwise result
Tasks
Share
Qwimi passed and base failed
23
7.67%
Base passed and Qwimi failed
19
6.33%
Both passed
225
75.00%
Both failed
33
11.00%
Total
300
100%
Among the 42 decisive tasks where only one model passed, Qwimi won 23/42 = 54.8%.
Category
Qwimi-only wins
Base-only wins
Both pass
Both fail
Coding
11
5
117
17
Tool-calling
4
3
63
5
Agentic
8
11
45
11
Total
23
19
225
33
300-task verdict: a slight overall quality win for Qwimi, driven by stronger coding and
structured tool-use performance, with a large reduction in end-to-end task time. Agentic
accuracy remains the clearest weakness.
8.2 Mixed 200-task benchmark
A separate 200-task run produced 400 raw answer rows. Categories with incomplete scorer or
export integration were omitted rather than counted as zero.
Model
Auto-scored rows
Passed
Pass rate
Average wall time
Base Qwen 3.6 27B Q6 MTP
151
46
30.46%
92.86 s
Qwimi 3.6 27B Coder Q6 MTP
166
51
30.72%
71.30 s
Overall change
Result
Absolute pass-rate gain
+0.26 percentage points
Relative pass-rate gain
+0.85%
Average wall-time reduction
23.2%
On the 146 tasks with a valid paired auto-score:
Pairwise result
Tasks
Share
Qwimi better
7
4.79%
Base better
2
1.37%
Same result
137
93.84%
Total paired tasks
146
100%
Among the nine decisive paired tasks, Qwimi won 7/9 = 77.8%.
Category
Base pass rate
Qwimi pass rate
Delta
HumanEval
80.00%
94.29%
+14.29 pts
Personal XML tool tasks
50.00%
100.00%
+50.00 pts
MBPP
5.77%
1.92%
-3.85 pts
Tool-calling checks
100%
100%
Tie
Architecture checks
100%
100%
Tie
Debugging checks
100%
100%
Tie
8.3 Interpreting the latency results
The measured advantage is task-level wall time, not proof of faster kernels or higher
per-token throughput.
Task time ≈ prefill time + generated tokens × decode latency per token
SFT can reduce task time by producing shorter, more direct, better-formatted outputs and
reaching the stop condition sooner. A controlled comparison of prompt throughput, generation
throughput, and generated-token counts is required to claim a per-token speed advantage.
8.4 Benchmark limitations
The published comparison used Q6 GGUF under llama.cpp, not MLX.
MLX Q4, Q6, and Q8 were not benchmarked against one another.
Wall time varies with Mac hardware, mlx-lm version, context length, cache behavior,
generation length, and sampling settings.
The custom suite is useful internal evidence, not a replacement for broad public benchmarks.
Agentic auto-scoring cannot capture every aspect of repository-level engineering quality.
9. Known limitations
Text-only specialization: the vision tower was frozen during SFT and was not improved.
Validated context: training and evaluation used a maximum sequence length of 16,384
tokens. Behavior beyond that length is unvalidated.
Agentic coverage: agentic data was much smaller than the coding portion, and the custom
benchmark showed a four-point agentic accuracy regression against the base model.
Quantization loss: this MLX conversion adds quantization error on top of BF16.
Apple Silicon only: MLX is designed for Apple silicon and is not a CUDA/Windows runtime.
Bleeding-edge architecture: Qwen 3.6 uses the qwen3_5 hybrid architecture with
GatedDeltaNet layers. Use a current mlx-lm build and verify generation locally.
MTP caveat: the name retains MTP from the source model, but this does not guarantee
that MLX preserves or actively uses the same MTP acceleration path as the GGUF benchmark.
Tool-calling caveat: XML-style tool output and server-side parsing can be version-sensitive.
Validate multi-turn tools before relying on them in an agent workflow.
Server caveat:mlx_lm.server is intended primarily as a local endpoint and is not a
hardened production service.
