This repository is the embedded-MTP edition. The compact imatrix language model is unchanged; the source model's single Q8_0 MTP block is appended as blk.40.* and explicitly tested on CUDA.

Files

File	Purpose	Size
`Ornith-1.5-35B-A3B-ZeroFuse-AD-IQ3_S-IQ3_XXS-MTP-Q8.gguf`	Language model with embedded Q8_0 MTP	16.41 GB / 15.27 GiB
`mmproj-Ornith-1.5-35B-A3B-uncensored-f16.gguf`	F16 vision projector	899 MB
`imatrix/imatrix.gguf`	Importance matrix used for the compact base quant	192 MB
`logs/quantize-AD-IQ3_S-IQ3_XXS.log`	Quantization log	123 KB
`tools/merge_mtp_gguf.py`	Minimal reproducible MTP merge tool	3 KB
`benchmarks/100k-recall.json`	Exact 100K recall result	544 B
`SHA256SUMS`	Artifact checksums	—

Download the model and mmproj for image input. The model alone supports text.

Verified structure

Property	Value
Parameters	35.51B
Decoder layers	40
MTP layers	1
GGUF tensors	753
`F32` tensors	310
`Q8_0` tensors	323
`IQ3_XXS` tensors	80
`IQ3_S` tensors	40

The MTP block contains 20 blk.40.* tensors copied byte-for-byte from the Junafinity Q8_0 source. Metadata is updated to block_count=41 and nextn_predict_layers=1.

Tested llama.cpp launch

Tested with the Muse llama.cpp branch. Builds without embedded draft-mtp support may load the model normally but will not provide speculative decoding.

bash

llama-server \
  --model Ornith-1.5-35B-A3B-ZeroFuse-AD-IQ3_S-IQ3_XXS-MTP-Q8.gguf \
  --mmproj mmproj-Ornith-1.5-35B-A3B-uncensored-f16.gguf \
  --device CUDA0 --split-mode none \
  --n-gpu-layers all \
  --n-cpu-moe 16 \
  --override-tensor 'blk\.40\..*=CUDA0' \
  --ctx-size 102400 --parallel 1 \
  --batch-size 256 --ubatch-size 64 \
  --threads 8 --threads-batch 8 \
  --flash-attn on \
  --cache-type-k q4_0 --cache-type-v q4_0 \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --spec-draft-device CUDA0 \
  --spec-draft-ngl all \
  --spec-draft-type-k q4_0 \
  --spec-draft-type-v q4_0 \
  --jinja --reasoning auto --reasoning-format deepseek \
  --ctx-checkpoints 1 --checkpoint-min-step 8192 \
  --cache-ram 0 \
  --host 127.0.0.1 --port 8082

--n-cpu-moe 16 is the tested 16 GB VRAM fit profile, not a universal recommendation. The MTP layer, target KV, draft KV, and MTP compute buffers were verified on CUDA0.

Local verification

Hardware: RTX 4090 Laptop GPU, 16 GB VRAM; i9-14900HX; Linux.

Gate	Result
8K bounded generation	pass
8K generation	31.79 tok/s
8K MTP acceptance	51.1%
100K exact recall	pass
100K prompt processing	149.74 tok/s
100K generation	32.57 tok/s
100K MTP acceptance	60.7%
Maximum observed VRAM at 100K	11,945 MiB

Exact 100K expected and returned value:

ORCHID-739184, COBALT-482017, SABLE-301955

These are single-machine deployment measurements, not general model-quality benchmarks. Prompt processing with embedded MTP was substantially slower than the best non-MTP deployment on this machine.

Reproducibility

The included merge script:

Reads the existing compact imatrix GGUF without requantizing it.
Selects only the 20 blk.40.* tensors from the Junafinity Q8_0 source.
Appends those tensors and updates the two MTP metadata fields.
Reopens the result and verifies tensor count, metadata, and an MTP tensor hash.

The included importance matrix contains 510 entries computed from 9,686 calibration chunks. The quantization log records the original tensor-level recipe and tool revision.

Credits and provenance

Base model: Ornith AI, Ornith-1.5-35B-A3B.
ZeroFuse abliteration and source GGUF: junafinity, using ZeroFuse. The source repository supplies the Q8_0 MTP tensors and vision projector.
Inference and quantization: llama.cpp and the tested Muse branch.
This compact quant and MTP packaging: l4zy0n3.

No claim is made that quantization or MTP packaging improves source-model intelligence. They trade precision and implementation complexity for local deployment fit and speculative-decoding capability.

License and responsible use

The immediate source repository declares Apache-2.0. The original Ornith model declares MIT. Follow all upstream license notices and terms. This model has had safety guardrails reduced or removed. Users remain responsible for legal, safe, and appropriate deployment.

l4zy0n3/Ornith-1.5-35B-A3B-ZeroFuse-AD-IQ3S-IQ3XXS-MTP-GGUF

Ornith 1.5 35B-A3B ZeroFuse AD-IQ3_S/IQ3_XXS MTP GGUF