This repository contains an experimental MTP-enabled build of Ornith-1.0-35B based on Unsloth's UD-Q4_K_XL GGUF.
The normal Ornith model weights and Unsloth quantization map are preserved. The only model-weight addition is a set of 20 MTP tensors (blk.40.*) grafted from the Q4_K_M donor-head package published by skinnyctax. The resulting model can use llama.cpp's draft-mtp self-speculative decoding without loading a separate draft model.
The donor file is not a standalone draft model. It contains only the MTP head tensors intended to be grafted into a compatible 40-block qwen35moe GGUF.
How this build was made
The build started from:
Ornith-1.0-35B-UD-Q4_K_XL.gguf
and grafted the Q4_K_M MTP donor heads:
ornith-1.0-35b-Q4_K_M-MTP-donor-heads.gguf
The transformation was performed with the donor repository's gguf_mtp_graft.py logic.
In other words, the original 40 model layers remain the target model and the appended layer is used as the MTP prediction head.
Tokenizer preservation
The tokenizer from the Unsloth base GGUF was preserved intact. Validation showed:
text
1vocab type = BPE
2n_vocab = 248320
3n_merges = 247587
This matters because the standalone donor-head GGUF intentionally does not contain the full tokenizer arrays and therefore cannot be loaded directly as --model-draft by current llama.cpp builds.
Important graft-script fix used for this build
The version of gguf_mtp_graft.py retrieved during this build required a small local correction when re-serializing general.architecture.
The published script stored that value with:
kv_pairs.append((key, 8, arch.encode("utf-8")))
For the GGUF string value to retain its length-prefixed encoding, this build used:
kv_pairs.append((key, 8, enc_str(arch)))
Without that correction, the generated file failed GGUF header parsing in the llama.cpp build used for validation. No model tensor data was changed by this fix; it only corrected serialization of that metadata string during header reconstruction.
Quantization details
This file should be understood as Unsloth UD-Q4_K_XL plus Q4_K_M MTP heads, not as a globally uniform Q4_K_M model.
The 733 original target tensors retain the quantization types from Unsloth's UD-Q4_K_XL GGUF.
The 20 appended MTP tensors retain their donor quantization.
No requantization of the base model was performed during the graft.
llama.cpp successfully created the MTP draft context against the target model, initialized speculative decoding, loaded the multimodal projector, and reached model loaded / listening state.
How to run with llama.cpp
Do not use the donor-head GGUF as --model-draft. The MTP heads are already embedded in this model.
For multimodal/vision use, obtain a compatible projector such as Unsloth's mmproj-F16.gguf from the original GGUF repository. The projector is not embedded in this model file.
Performance note
The donor repository reports substantial MTP acceptance and inference-speed improvements for its own Q4_K_M and Q6_K grafted builds. Those measurements should not be assumed to apply directly to this UD-Q4_K_XL build.
This specific file has been functionally validated for correct loading and MTP initialization, but no controlled acceptance-rate or tokens-per-second benchmark is claimed here yet.
Credits
DeepReinforce AI — Ornith-1.0-35B base model
Unsloth — GGUF conversion and UD-Q4_K_XL quantization
skinnyctax — MTP donor-head extraction, grafting approach, and graft script
llama.cpp — GGUF runtime and draft-mtp speculative decoding support
This repository does not claim authorship of the original model, quantization method, or donor MTP work. It packages the specific UD-Q4_K_XL + MTP graft described above for convenient llama.cpp use.
