Q8_0 GGUF quantization of ahalt/qwen3-event-extraction-exp5.1, the attribute-extraction model for NGEC (Next Generation Event Coder). See that repo's model card for what the model does, its evaluation numbers, and the full prompt format.

Built with convert_hf_to_gguf.py --outtype bf16 followed by llama-quantize ... Q8_0 (8.5 bits/weight). No importance matrix was used — at Q8_0 the quantization is close enough to lossless that it isn't needed; see llama.cpp's quantize docs.

The chat template is embedded in the GGUF, so llama-server's /v1/chat/completions endpoint applies it the same way tokenizer .apply_chat_template(..., enable_thinking=False) does on the Python side. Use the same system prompt and message format documented on the base model's card — the ## Document: … ## Event Type: <definition> v5 format, not the format the older ahalt/event-attribute-extractor GGUFs use:

Within the NGEC package, ngec.attribute_model.AttributeModel(backend="llamacpp") builds this request for you — point llamacpp_url at a running llama-server and model_name/NGEC_ATTRIBUTE_MODEL at ahalt/qwen3-event-extraction-exp5.1 so the prompt format is picked correctly (the GGUF supplies weights; the Python side still needs the tokenizer/prompt-format pairing). See demo/deploy/README.md for a supervised deployment of llama-server on a CPU host.

Q8_0 does not reproduce the unquantized model byte-for-byte, but on the demo's curated example set it produces valid JSON on every prompt and matches the reference (bf16, served via vLLM) closely — see demo/DESIGN.md's "Two backends" section in the NGEC repo for the comparison methodology.