Views
No views yet
tokenizer.json so the volume is self-contained.qwen3.5-0.8b-gguf: the GGUF carries the
model's trained multi-token-prediction head (one extra layer), and eyesoff-ai's
"draft": "mtp" setting uses it for speculative decoding - the head proposes
the next few tokens at near-zero cost and the trunk verifies them in one pass.
Output is byte-for-byte the trunk's own (exact-match verification); the head
only changes speed. It is the smallest self-drafting volume in this org, and
the cheapest way to stand up a chat deployment that still decodes fast.qwen3.5-0.8b-gguf, and the two are not interchangeable.
That volume exists to be a draft for qwen3.5-9b-gguf, qwen3.6-27b-gguf and
qwen3.5-122b-gguf; in that role a draft model's own MTP head is never
consulted (the target does the verifying), so attaching this volume as a
draft buys nothing and costs the extra head's weights. Use this one when the
0.8B is the model being served, and the plain one when it is drafting for
something bigger.mmproj files and no
tokenizer.json. This volume carries ONLY the one served quant plus the
tokenizer (same rationale as the sibling volumes): a multi-gguf volume makes the
host's preload pick ambiguous, and the extra files never serve.UD-Q4_K_XL would save ~0.26 GB, which is noise against
any share this model would be deployed on, and 4 bits is where quantization
damage starts to show on a model this small. It matches the Q8_0 the sibling
qwen3.5-0.8b-gguf serves.Qwen3.5-0.8B-Q8_0.gguf) but the bytes differ - this one is ~22 MB larger and
hashes differently, because it carries the extra head. Volumes mount at
/models/<volume-name>/ so the two never collide in a guest, but do not assume
a file of that name is the plain build; check the sha256 below.| File | Upstream | Revision | sha256 |
|---|---|---|---|
Qwen3.5-0.8B-Q8_0.gguf | unsloth/Qwen3.5-0.8B-MTP-GGUF | cf8a611f6ed2c2060046219a19f12cd3d5ecd67c | c54f8b67069c70085b98440de696b44da8250250ac69a961b41133def876e262 |
tokenizer.json | Qwen/Qwen3.5-0.8B | 2fc06364715b967f1860aea9cf38778875588b17 | 5f9e4d4901a92b997e463c1f46055088b6cca5ca61a6522d1b9f64c4bb81cb42 |
qwen3.5-0.8b-gguf,
qwen3.5-9b-gguf, qwen3.6-27b-gguf, qwen3.6-27b-mtp-gguf and
qwen3.5-122b-gguf (the Qwen3.5/3.6 family shares it - same sha256, vocab
248320, <|im_end|> 248046 / <|endoftext|> 248044), which is exactly the
property the speculative-decoding compatibility gate requires.qwen35, hybrid-SSM trunk + 1 dense-attention MTP
layer): 24 trunk layers, full_attention_interval 4 (6 trunk KV layers), 2 KV
heads, head_dim 256, trained context 262144. For VRAM budgeting count the MTP
head as one more full-attention KV layer (7 total) - its context is created
lazily when a speculative session first opens. The GGUF reports
block_count 25 and nextn_predict_layers 1; the head is blk.24, carrying
the nextn.eh_proj / enorm / hnorm / shared_head_norm tensors.image-text-to-text) and its projector
is a qwen3vl_merger, but this volume is TEXT-ONLY: no mmproj is bundled and
the catalog entry does not set vision. qwen3-vl-8b-gguf is the vision volume
in this org.LICENSE is the Apache-2.0 text; both upstreams are Apache-2.0./models/qwen3.5-0.8b-mtp-gguf.