An INT4 ONNX package that runs on CPU with the nxrt Python API. It demonstrates onnx-genai's canonical inference_metadata.yaml format, which is the authoritative contract in this package. genai_config.json is retained only for ecosystem compatibility.

The metadata is a fully serialized pipeline.workflow. The autoregressive loop is data, not runtime-specific code: the decoder graph and the ten token-policy graphs under policies/ are declared as workflow components, all 28 key/value cache pairs are threaded through serving.state_service groups, and sampling, termination, and length bookkeeping are ONNX graphs the runtime executes like any other component. The package therefore runs on the generic workflow runtime with no decoder-specific lowering step.

There is no model.io section. That legacy form described a single decoder in a shape only a special-cased decoder loader could execute; it is not supported.

Verified with nxrt==0.1.0.dev3 and CPUExecutionProvider:

ONNX_GENAI_KV_MAX_LEN limits the shared KV allocation for this short example; the model metadata preserves the full 131,072-token context limit.

Derived from deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B. The model is distributed under the MIT License and is distilled from Qwen2.5-Math-1.5B, which is distributed under Apache-2.0. See LICENSE and the upstream model card.

Review inference_metadata.annotated.yaml for inline explanations of this package's workflow, tensor/state/cache contracts, and fail-closed omissions. inference_metadata.yaml remains the canonical machine-authored contract; automated validation confirms both files parse to the same metadata object.