What this is: the fraQtl calibration sidecars for Mistral-Nemo-Instruct-2407 (12B, 40 layers). Built by the fraQtl sidecar factory with zero kernel changes — the same membrane runtime, receipts below.

Decode tok/s, single sequence:

Concurrent capacity ("users" = N parallel sequences via -np N, each with an independent KV stream and a full 32,832-token context allocation; exact command line in every receipt JSON):

We publish Nemo receipts to 32K. At 128K, needle retrieval fails on this model under EVERY configuration we tested — including llama.cpp q8_0 KV and stock upstream llama.cpp with uncompressed fp16 KV (both 0/1). The model's advertised 128K window exceeds its effective retrieval range; this is a property of the model, control-verified, not of the compression. The membrane matched fp16 retrieval behavior exactly at every context.

Additional disclosure: isolated single-needle misses at high load — q8_0 dropped 1/16 at 16 users; membrane dropped 1/28 at 28 users (and passed 32/32 at 32 users). Both arms show occasional high-load misses on this model; per-cell counts are in the receipts.

Same prebuilt runtime and flags as the Qwen3-4B sidecar card, with this model's sidecar files (this repo). Runtime binaries (Linux x86_64, CUDA, SM80+SM90, manifest-sha'd): fraQtl/fraqtl-membrane-llamacpp-runtime

The on-device lane — calibration-aware Hi-Fi GGUFs with KLD receipts: org page. The same membrane in vLLM: nine concurrent ≈128K users on one A100 (134.1 tok/s, 9/9 retrieval) — receipt.