No weights are republished here. This is a measurement report: IBM's official granite-4.0-h-small-GGUF files run on xyntetik-runner as of v0.1.20-alpha+, and this repository carries the reproducible evidence and the measured-envelope manifest for the exact file measured, bound by SHA-256.

Serve with `xyntetik-runner` — tool calls that still parse when the budget runs out

This GGUF is built to be served with xyntetik-runner, a single-binary, dependency-free C11 inference engine (CPU / CUDA / Metal, OpenAI-compatible server). Its one differentiator you cannot get from another engine:

When a tool call runs past its token budget, Runner closes it to the smallest schema-legal document, so the arguments still parse.

Most OpenAI-compatible engines return finish_reason: "length" with an empty or malformed tool_calls the caller has to repair or retry. Runner does forced-truncation recovery: once a JSON document starts, it emits a legal ending when the budget expires. On local models — tight context, slow generation — that is the difference between an agent loop that finishes and one that retries from scratch.

Measured — what each engine hands the caller when the token budget cuts a tool call short (same box, same tool schema, same prompt, tool_choice:"required", temperature 0, budgets 1→64):

engine	budget too small (1–16 tokens)	enough budget (64, control)
Runner	executable `tool_calls`, arguments parse	completes
vLLM 0.27.1	no call; protocol framing leaks into `content`	completes
llama.cpp b10488	no call; leak, then `tool_calls` with unparseable args	completes
Ollama 0.32.14	no call; empty content, then HTTP 500	completes
TensorRT-LLM 1.2.1 †	no call; `<tool_call>` leak, then empty content	completes
SGLang 0.5.17 †	no call; `<tool_call>` leak, then empty content	completes

Every engine completes at 64 tokens — the failure is truncation, not misconfiguration. Below that, only Runner returns an executable call. This is the behaviour across every OpenAI-compatible engine measured, not a claim about engines not measured. † TensorRT-LLM and SGLang were measured on a Qwen3-1.7B substitute; truncation recovery is a property of the runtime, not the model.

Constrained decoding also holds the shape of a tool call under quantization: on a full quant ladder, schema conformance and tool selection stayed 100% down to Q4_0 (argument agreement decayed to 50%) — it guarantees the shape of a call at any quantization, not its contents.

Quickstart

bash

# build the single binary (no deps), then serve this GGUF:
runner -m granite-4.0-h-small-Q8_0.gguf --serve
# point any OpenAI-compatible client at http://localhost:8080/v1

Full recipe, raw responses, and the per-release regression gate: the truncation benchmark.

Runs on `xyntetik-runner` (granitehybrid hybrid)

granite-4.0-h-small is a hybrid Mamba-2 / attention MoE model — architecture granitehybrid: a selective-SSD (state-space) recurrence interleaved with GQA attention, a routed MoE FFN plus an always-on shared expert on every layer, and the four granite muP scalars. Runner implements the full CPU path: per-token Mamba-2 decode, a chunked-scan prefill (bit-identical to the serial sweep, ~1.8× prompt throughput), and the recurrent-state cache seam (the fold is snapshotted / restored on a rewind and stored beside the KV in the prefix cache, so an exact prompt-prefix hit restores it in a memcpy). Runs on CPU: the Mamba-2 SSD scan itself now has a CUDA device kernel, but this family's shared-expert MoE has no device path yet, so the backend falls back to CPU there rather than computing part of the model wrong.

runner -m granite-4.0-h-small-Q8_0.gguf --serve

Greedy identity vs llama.cpp — measured, not asserted

Reproducible greedy comparison via the committed scripts/reference_compare.py (both engines served, exact generated UTF-8 from /v1/completions at temperature 0; 5 prompts × 32 tokens, CPU):

prompt class	result
factual ("The capital of France is")	byte-identical
counting ("1 2 3 4 5 6 7 8")	byte-identical
story ("Once upon a time…")	byte-identical
open-ended code (`def fibonacci(n):`)	diverges after a shared prefix (near-tie)
open-ended JSON continuation	diverges after a shared prefix (near-tie)

3/5 byte-identical. The two divergences are greedy cascades from a near-tie on open-ended continuations — the same quantization-noise-floor envelope documented for gpt-oss and qwen3moe, not wrong math: runner and the oracle rank the same top candidates, and every high-confidence prompt matches exactly. Raw JSON: ssm-greedy-reference-2026-08-20/ (runner 0.1.20-alpha+, reference llama.cpp ea12b27).

Measured file (SHA-bound)

granite-4.0-h-small-Q8_0.gguf — sha256 232b8a426a6d6ab153752f3cf52d7ef9db42e2bdc9c854fb7bd4cdd15e292ad7 — from ibm-granite/granite-4.0-h-small-GGUF.

An envelope manifest (granite-4.0-h-small-Q8_0.gguf.envelope.json) is included: place it beside the GGUF and runner reports the measured state at load. Verdict: experimental — a dated measurement, not a standing quality claim. Its reported-only tool_calling block records: truncation-recovery holds 6/6 (an engine property, measured on a proxy model), native tool protocol generic (granite has no native tool grammar; it gets runner's constrained JSON envelope — which is exactly what the truncation guarantee covers).

Model, attribution, and license

Base model: ibm-granite/granite-4.0-h-small by IBM (Apache-2.0). All model weights, capabilities, and credit belong to IBM; this repository republishes no weights — only measurements of IBM's official GGUF release.
Not affiliated with, sponsored by, or endorsed by IBM.

Joakimpalm-Zen/granite-4.0-h-small-runner-report

granite-4.0-h-small on xyntetik-runner — measurement report (granitehybrid)

Serve with xyntetik-runner — tool calls that still parse when the budget runs out

Quickstart

Runs on xyntetik-runner (granitehybrid hybrid)

Greedy identity vs llama.cpp — measured, not asserted

Measured file (SHA-bound)

Model, attribution, and license

Joakimpalm-Zen/
granite-4.0-h-small-runner-report

granite-4.0-h-small on `xyntetik-runner` — measurement report (`granitehybrid`)

Serve with `xyntetik-runner` — tool calls that still parse when the budget runs out

Runs on `xyntetik-runner` (granitehybrid hybrid)