Views
No views yet
--mmproj loaded.| faithful | abliterated | delta | |
|---|---|---|---|
| MTP draft acceptance | 87.4% | 86.4% | −1.0 pp |
| tokens/s (gate) | 49.1 | 44.1 | −5.0 |
token_embd excluded (it is a get_rows, not a matmul).
It is a measure of where the bits go, not a measured quality score.
Every file size matched its pre-build prediction to 0.01 GB.| tier | GB | active bpw | file bpw |
|---|---|---|---|
MTP-UD-Q6_K | 30.21 | 8.063 | 6.804 |
MTP-UD-Q5_K_S | 25.84 | 7.693 | 5.820 |
MTP-UD-Q4_K_XL | 23.22 | 7.471 | 5.230 |
MTP-APEX-I-Balanced-v2D-lite | 26.30 | 6.913 | 5.922 |
MTP-UD-IQ4_XS | 18.68 | 6.762 | 4.209 |
MTP-APEX-I-Quality-v2D-lite | 23.85 | 6.699 | 5.371 |
MTP-APEX-I-Compact-v2D-lite | 17.57 | 5.228 | 3.956 |
MTP-APEX-I-Mini-v2D-lite | 14.37 | 4.180 | 3.208 |
blk.40 — it is grafted after calibration and never runs during an imatrix
pass — so it is unguided RTN either way, and draft acceptance converts directly
into tokens/sec.UD-IQ4_XS (18.68 GB) scores above
APEX-I-Quality-v2D-lite (23.85 GB) on the active path, and UD-Q4_K_XL (23.22 GB)
above APEX-I-Balanced-v2D-lite (26.30 GB). The UD maps pin attention, shared
experts and token_embd at Q8_0 at every tier, and roughly 55% of active
parameters are non-routed — attention, the shared expert and the 508 M-param output
head all run on every token, while routed experts contribute 8/256. If you are
choosing on quality per GB, start from the top of this table, not from file size.mtp.* tensors into
safetensors and then converts, which was impossible, so the graft was done one
stage later, directly at GGUF level:+ 20 blk.40.* tensors, copied byte-for-byte from the faithful build's bf16 master
~ qwen35moe.block_count 40 -> 41
+ qwen35moe.nextn_predict_layers (absent) -> 1sha256 faac91f15cbe54475faa2578bedc46a7c29a947b8a3e7ef3ecd376ae079826ab. So this
repo provably carries the same head as the faithful one.ffn_gate_inp_shexp as [2048, 1], convert_hf_to_gguf.py writes [2048] — the
same 2048 values). A grafted-master gate then asserted 753 tensors, 20 in blk.40,
block_count=41, nextn_predict_layers=1.Huihui-Ornith-1.0-35B-abliterated-mmproj-F16.gguf (0.90 GB) is required —
without it you have a blind model. It is huihui-ai's own projector, kept separate
so one copy serves all eight tiers.1llama-server -m Huihui-Ornith-1.0-35B-abliterated-MTP-UD-Q5_K_S.gguf \
2 --mmproj Huihui-Ornith-1.0-35B-abliterated-mmproj-F16.gguf \
3 -c 8192 -fa on --jinja \
4 --spec-type draft-mtp,ngram-mod \
5 --spec-draft-n-max 1 --spec-draft-n-min 0 --spec-draft-p-min 0.75 \
6 --spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 24 --spec-ngram-mod-n-match 48--mmproj silently force-disables ctx_shift and cache_reuse.calibration_datav5 + unsloth, 2463 chunks,
1.26 M tokens) used by the faithful build. Abliteration changes activation
statistics, so a base imatrix will misallocate precision to some degree, and on a
256-expert MoE it can misdescribe routing too. This was a deliberate trade to
keep the build on a CPU box and to hold every variable except the trunk constant.
It is the one known-suboptimal ingredient here.code-novel + copy-edit
prompt mix. Comparable to the faithful build's 87.4% because it is the identical
harness — not comparable to acceptance figures quoted from other harnesses.BF16/ — the abliterated bf16 + MTP master (71.07 GB, 753 tensors), so any
future tier rebuilds from it with no graft and no conversion.sha256sums.txt