NVFP4 (4-bit e2m1, per-16 FP8-e4m3 scales) GGUF of ornith-ai/Ornith-1.5-9B, quantized from the official BF16 safetensors release (token embeddings and output head Q5_K, norms F32). 18.8 GB BF16 → 5.4 GB. Built on request (discussion) as the 9B sibling of Ornith-1.5-35B-A3B-NVFP4-MTP-GGUF.

The MTP head in this file is a graft, and the card says so. The official Ornith-1.5-9B checkpoint declares an MTP head in its config (mtp_num_hidden_layers: 1) but ships no mtp.* tensors — 760 tensors where its base has 775. This artifact takes the trained MTP head from Qwen/Qwen3.5-9B (Apache-2.0, the base model this fine-tune derives from — the two checkpoints are tensor-census-identical except for those 15 head tensors) and grafts it onto the unchanged official Ornith trunk at blk.32 (nextn_predict_layers=1). A draft head can never change output — the target verifies every drafted token — it only moves acceptance and speculative speed. Measured acceptance with the grafted head is in the tables below; the speculative stream is gated token-identical to plain greedy decode at every K.

Built as a serving artifact for memra, a from-scratch Rust + CUDA inference engine for RTX Blackwell (sm_120a) with per-request exactness gates: speculative, graphed, and batched serving are gated byte-identical to plain decode.

NVFP4 is not an upstream llama.cpp tensor type: this file runs on memra and on the NVFP4 branch of avifenesh/llama.cpp. For upstream-llama.cpp/Ollama use, the official BF16–Q8_0 GGUFs are the right pick (they carry no MTP head).

Engine: https://github.com/avifenesh/memra (MIT, crates.io: memra-server)
Hosted inference: https://inference.tiyuvta.ai
Context: 262,144 native; chat template embedded — byte-identical to the repo's chat_template.jinja (sha256 9dd2fbd270fe…44e8fd235; XML tool calling + <think> reasoning)
Vocabulary: 248,320 (text→text serving; the upstream vision tower is not in this artifact)

Provenance and verification


Trunk	`ornith-ai/Ornith-1.5-9B` official BF16 safetensors (MIT), unchanged bytes; per-shard sha256 receipts banked at mint
MTP head	the 15 `mtp.` tensors of `Qwen/Qwen3.5-9B` (Apache-2.0, trained by Qwen for this backbone), grafted at `blk.32` through the standard converter mapping; per-tensor sha256 receipts banked at mint. Ornith-1.5-9B itself ships no `mtp.` weights
Conversion	`convert_hf_to_gguf.py` (avifenesh/llama.cpp `nvfp4-imatrix-scale-search`), BF16, MTP bundled (`qwen35`, 33 blocks incl. NextN, `rope.dimension_count 64`)
Quantization	`llama-quantize` NVFP4 ftype (same branch), `--output-tensor-type q5_k --token-embedding-type q5_k`: blocks + MTP NVFP4, embeddings/output Q5_K, norms F32 — the same recipe as the 35B sibling
Main file	`Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf`, 5,442,334,016 B, sha256 `7a8631ecdffe51f90533f4fa053a6dfb96528d4240a9ed81abd1c0c087c47502`
Exactness	memra v0.104.1 batteries, 2026-08-22, RTX 5090 Laptop: `kernel-check` green · `run-spec` K=1..8 self-consistency PASS every K (spec ≡ plain greedy, token-identical), raw and chat-templated, embedded and masked head · prefill ≡ decode argmax and batched-prime ≡ tokenwise argmax MATCH on all probes · memra chat render ≡ `transformers.apply_chat_template` prompt ids on all probes · 48-tok greedy vs BF16 CPU oracle forks only at logit near-ties (top-2 margin ≤ 0.5, rig token = oracle rank-2; an official-Q8_0 control forks from the same oracle within 30–40 tokens on the same probes) · serve surface: greedy chat determinism ×2 byte-identical, tools round-trip emits OpenAI `tool_calls` with schema-typed arguments (`"celsius":true`) and zero content leak, per-request `usage.spec` accounting

Measured acceptance (exactness harness, not a throughput bank)

Greedy run-spec, memra v0.104.1, RTX 5090 Laptop. Acceptance = accepted/drafted over the run; the speculative stream is token-identical to plain decode in every row (that is the gate). This laptop card is thermally limited, so no tok/s numbers are published here — measure on your own hardware (run-spec, or frspec-owngen --validate).

head	probe	K=1	K=2	K=3
embedded grafted head	chat-templated code prompt, 64 tok	0.80	0.71	0.59
embedded grafted head	raw continuation probe, 32 tok	0.72	0.57	0.45
masked head (`frspec-sxc32768`)	chat-templated code prompt, 64 tok	0.70	0.71	0.56

What ships here, file by file

file	size	what it is
`Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf`	5.4 GB	The trunk. NVFP4 weights, Q5_K embeddings + full 248,320-row LM head, grafted MTP (NextN) block at `blk.32`. Serves on its own; every other file in this repo is optional speed.
`mtp-Ornith-1.5-9B-NVFP4-frspec-sxc32768.gguf`	0.94 GB	Standalone masked MTP draft head, top-32,768 own-generation ranks (agentic-session prompt corpus).
`ornith15-9b-ranks-sxc-32768.gguf.txt`	~186 KB	Ranks list, plain text: one token id per line, most-frequent first, 32,768 lines. Drives memra's load-time trim (`MEMRA_FRSPEC_TRIM`) — this is the file that pairs with a trunk that already carries a head.
`ornith15-9b-ranks-sxc-32768.gguf`	~131 KB	The same ranks as a one-tensor GGUF container (`d2t`, i32 `[32768]`). Interchangeable with the `.txt` anywhere a ranks file is accepted.

Where the masked head lives, and what the mask does

mtp-Ornith-1.5-9B-NVFP4-frspec-sxc32768.gguf contains:

tensor	shape / type	role
`output.weight`	`[4096, 32768]` NVFP4	the masked LM head — 32,768 rows gathered from the trunk's 248,320-row head, in rank order
`d2t`	i32 `[32768]`	the mask itself: `d2t[i]` = full-vocab token id of masked row `i`
`blk.32.nextn.{eh_proj,enorm,hnorm,shared_head_norm}.weight` + `blk.32.attn_` / `blk.32.ffn_`	—	the MTP (NextN) block, extracted byte-verbatim from the trunk, block requantized Q4_K_M
`token_embd.weight`	`[4096, 248320]` Q5_K	full-vocab embeddings (drafting reads the trunk's ids)

"Masked head" means: the draft proposes tokens only from the top-32,768 ids ranked by how often this model itself emits them (FR-Spec-style d2t trim). The ranked distribution is 100% model-generated — 138,065 own-generated tokens over real agentic-session prompts; external text was used as prompts only. topN = 32768 fixes the masked head's shape — a different topN is a different artifact.

The mask can never change output. Verification runs on the target's full vocabulary, so a trim moves draft acceptance (speed) and nothing else — run-spec asserts the masked-head speculative stream token-identical to plain decode, and that gate is PASS on this artifact (file sha256 6b027b05ba92dfda2f9786a1a970b028b8475345223b9dc85b25d1e51224775b). On the build's --validate A/B the trim cost 0.5 acceptance pts against the untrimmed embedded head (0.429 vs 0.434 over the held-out sweep) for a 7.6× smaller draft-head read — verdict GOOD on the measurement card; end-to-end tok/s decides it on yours.

How to run it (memra)

Install memra per its README (release installer or cargo build --release). Three ways to draft, all gated byte-identical to plain decode:

1. Trunk alone — the grafted head is embedded

bash

MEMRA_COMPAT=openai \
MEMRA_MODELS="ornith9b=Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf" \
memra-server

Paths accept hf:owner/repo[:file-substring] specs, which download on first use:

MEMRA_MODELS="ornith9b=hf:tiyuvta/Ornith-1.5-9B-NVFP4-MTP-GGUF:Q5K-mtp" memra-server

2. Trunk + pre-trimmed masked head

bash

MEMRA_MODELS="ornith9b=Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf+mtp-Ornith-1.5-9B-NVFP4-frspec-sxc32768.gguf" \
memra-server

One-off CLI (no server) — run-spec sweeps K and asserts the speculative stream is token-identical to plain decode:

bash

MEMRA_MTP_DRAFT=mtp-Ornith-1.5-9B-NVFP4-frspec-sxc32768.gguf \
run-spec Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf

3. Ranks file only — the trim happens at load

MEMRA_FRSPEC_TRIM=<ranks> takes either the .txt or the ranks .gguf and self-trims at load: the engine gathers the 32,768 ranked rows from the trunk's own output.weight bytes (byte-level row gather, zero requant).

bash

MEMRA_FRSPEC_TRIM=ornith15-9b-ranks-sxc-32768.gguf.txt \
MEMRA_MODELS="ornith9b=Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf" \
memra-server

Note on safetensors trunks: the official Ornith-1.5-9B safetensors checkpoint ships no MTP head, so there is nothing there for a ranks file to trim — speculative drafting for this model comes from this repo's GGUFs.

The attach is a log line, not the absence of an error

A wrong path or flag does not fail — the trunk's embedded full head drafts instead. Check the boot log:

you used	the line that proves it
`+draft` in `MEMRA_MODELS`	`[worker] ornith9b: regime draft attached (…frspec-sxc32768.gguf)`
`MEMRA_MTP_DRAFT`	`[mtp-draft] external draft head: blk.32, source=output.weight, head_vocab=32768 (trimmed, d2t map)`
`MEMRA_FRSPEC_TRIM`	`[frspec-trim] self-trimmed head: 32768 rows of main output.weight (Q5_K)`

Other runtimes

The trunk is a standard GGUF layout for the qwen35 architecture with a NextN block; engines that read NextN heads see the grafted MTP out of the box. The masked draft files carry the d2t tensor layout (see ggml-org/llama.cpp#25187); outside memra, draft from the trunk's embedded full head instead of the masked files.

Build your own ranks (and your own masked head)

Rank files are vocab + distribution artifacts of the exact serving model: derive fresh ranks from the model's own generations for every model and every requant. The tools ship in the memra repo:

bash

# 1. Ranks from the model's OWN generations (corpus text is prompts only).
frspec-owngen Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf myranks.gguf 32768 my-prompts/
# -> myranks.gguf (d2t container) + myranks.gguf.txt (one id per line)

# 2. Extract the MTP block byte-verbatim, trim the head to your ranks, requantize
#    (NVFP4 head + Q4_K_M block — the measured-best order). Needs python3 + numpy,
#    llama.cpp's gguf-py (MEMRA_GGUFPY) and llama-quantize (MEMRA_QUANTIZE).
tools/make-trimmed-draft.sh Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf myranks.gguf.txt my-draft.gguf 32768

The measured laws (docs/DRAFT-REGIME.md): per model, per requant (foreign ranks measured −12 acceptance pts on an identical tokenizer); chat template ON if you serve chat (a raw-derived rank set once cost −15 acceptance pts); corpus floor ≥ 4× topN own-generated tokens (131,072 for a 32,768 head); validate before trusting (frspec-owngen … --validate A/Bs trimmed vs untrimmed end-to-end).

Built with convert_hf_to_gguf.py + llama-quantize (NVFP4 branch) + frspec-owngen + tools/make-trimmed-draft.sh — the same recipe documented above.

Avifenesh/Ornith-1.5-9B-NVFP4-MTP-GGUF

Ornith-1.5-9B — NVFP4 GGUF with MTP head (memra serving artifact)

Provenance and verification

Measured acceptance (exactness harness, not a throughput bank)

What ships here, file by file

Where the masked head lives, and what the mask does

How to run it (memra)

1. Trunk alone — the grafted head is embedded

2. Trunk + pre-trimmed masked head

3. Ranks file only — the trim happens at load

The attach is a log line, not the absence of an error

Other runtimes

Build your own ranks (and your own masked head)

Avifenesh/
Ornith-1.5-9B-NVFP4-MTP-GGUF