Measured/structural per-tensor bit allocation (APEX) GGUF quants of meta-llama/Llama-4-Scout-17B-16E-Instruct, built entirely from a full-precision BF16 GGUF (no safetensors ever downloaded — see Method below).

Llama 4 Scout was trained with up to a 10,485,760-token (10M) context window. If you launch llama-cli / llama-server without an explicit --ctx-size, llama.cpp defaults the KV cache to the model's own trained context length — not a small sane default. For most models that's just an oversized-but-harmless KV cache; for this model it means an attempt to allocate a KV cache sized for 10 million tokens, which can consume many terabytes of memory and hard-lock a machine (this happened to us during testing — full physical reboot required, not a clean OOM kill).

Always pass --ctx-size sized to what you actually need and what your hardware can hold, e.g.:

8K–32K is plenty for most chat/tool-use workloads. Only reach for six- or seven-figure context sizes if you have the RAM/VRAM to back it, and scale up deliberately — never rely on the default.

FYI, these models are big and I'm on Starlink. I'll upload models as I can.

token_embd.weight and output.weight are held at Q6_K in all three tiers.

Validation status: allocation is structural (role/depth-band priors), not measured — Scout is too large for this pipeline's current measured-sensitivity mode (which needs the whole model resident in memory for per-tensor forward passes; a memory-bounded version is future work). Real-generation coherence and tool-calling checks are pending a re-run and not yet reflected in this card — an in-progress gate run for i-mini was interrupted by the context-size incident described above, before the fix landed. Treat these quants as built-and-untested until that gate run completes; this README will be updated with results.

No wikitext PPL-ratio baseline: the fp BF16 model (~205 GiB) doesn't fit resident on the build hardware (119 GB unified memory) to run a reference perplexity pass.

Built without ever downloading Scout's ~217 GB of raw safetensors. The importance matrix (imatrix) was computed by a band-serialized generator that reads weights directly from a pre-converted full-precision BF16 GGUF (unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF), materializing one transformer layer at a time — same approach validated byte-exact against the safetensors path on a smaller model first (granite-3.1) before trusting it on Scout. This keeps peak memory bounded regardless of total model size, which is what made a 109B-parameter (17B active) MoE tractable on a single machine with 119 GB of unified memory.

Bit allocation is structural: attention and shared-expert tensors get higher precision than routed expert tensors, following role/depth-band priors rather than per-tensor measured sensitivity.

Imatrix computed over Salesforce/wikitext, 126 chunks × 512 tokens — the same calibration corpus used across this project's other APEX quants.

Unofficial community quantization; not affiliated with or endorsed by Meta or the base model's publisher. Use of this model is subject to the Llama 4 Community License Agreement linked above.