poolside's 117B sparse-MoE coding model served on one 128 GB DGX Spark at 6.0 bits per parameter with the full 256k context window, through stock vLLM with an out-of-tree plugin — no forked runtime, no core patches. Weights are stored in the gridbook codebook formats (FP8-CB / NVFP4-CB: product-VQ over 8-weight vectors whose codebook values live on hardware grids) and decoded inside custom CUDA/Triton kernels at matmul time. Formats + kernels: github.com/RobTand/gridbook. Produced by the PrismaQuant measured-allocation pipeline.

Base model: poolside/Laguna-S-2.1 (OpenMDW-1.1), 48 layers, 256 routed experts top-10 + 1 shared expert, hybrid sliding(512)/full attention, 256k context.
Artifact: single safetensors file, 84 GB, 6.009 bpp over quantizable parameters. Allocation chosen on a measured 27-point rate-distortion sweep (knee analysis included in the repo discussion), per-Linear formats solved by knapsack DP over per-(Linear, format) measured costs.
Measured on-box (DGX Spark GB10, vLLM eager, fp8 KV, plugin 2026-07-23, serve command below): decode 14.9 tok/s, prefill 1,821 tok/s @8k / 1,822 tok/s @63k; 256k context boots; exact needle retrieval at 63k-token depth. Only 12 of 48 layers are full-attention (the rest slide at w=512), so a full 256k request costs ~6 GiB of fp8 KV — the whole model + 256k cache fits the Spark's unified pool. Config tradeoff: the fast-prefill command below (--max-num-batched-tokens 16384, util 0.85) leaves 1.53× full-256k request concurrency; default batching at util 0.87 gives 3.28× concurrency with ~6× slower long-prompt prefill. A DFlash drafter adds nothing on this hardware (37.5% acceptance; measured net-negative to parity) — serve drafter-free.

Format breakdown (measured allocation, not hand-picked)

format	where	params	bytes
FP8_CB_K48 (6.0 bpw)	60 expert units	72.5B	54.4 GB
FP8_CB_K44 (5.5 bpw)	22 expert units	26.6B	18.3 GB
FP8_CB_K40 (5.0 bpw)	10 expert units	12.1B	7.5 GB
FP8_CB_K38 (4.75 bpw)	2 expert units	2.4B	1.4 GB
FP8_CB_K47 / NVFP4_CB	27 small Linears	0.3B	0.2 GB
BF16	attention / shared-expert / dense mlp	3.0B	6.0 GB

The experts carry the compression, graded down the fp8-CB ladder by measured sensitivity; the entire per-token dense path (attention, gates, shared expert) rides BF16 — at this budget those 3B params cost only 6 GB and buy maximum fidelity on every high-leverage tensor. This heterogeneous assignment is the PrismaQuant thesis: allocation measured end-to-end beats any uniform format.

Serve this model


Plugin	`gridbook` — an out-of-tree vLLM quantization plugin. Stock vLLM, no fork, no core patches.
GPU	NVIDIA Blackwell, compute capability `sm_120` / `sm_121`. Measured on GB10 / DGX Spark (`sm_121`). On older GPUs the plugin still loads but runs its Triton fallback kernels — correct, not fast, and not a production serving target.
Memory	89.4 GB of weights on disk = 83.2 GiB resident. Needs a ~128 GB unified/VRAM pool: measured on one DGX Spark at `--gpu-memory-utilization 0.85`. Only 12 of 48 layers are full-attention (the rest slide at w=512), so a full 256k request costs ~6 GiB of fp8 KV — model + 256k cache fit the Spark's pool.
vLLM	Version used for these measurements is not recorded on this card.
Toolchain	CUDA toolkit with `nvcc` on `PATH` in the serving container — the plugin JIT-builds its kernels on first model load (~30 s, cached). `nvcc` 13.0 is the tested toolchain.
Parallelism	Single GPU (`tp=1`). Tensor parallelism is not implemented in the plugin.

bash

pip install gridbook
vllm serve rdtand/Laguna-S-2.1-prismaquant-gridbook-6bit-vllm \
  --host 0.0.0.0 --port 8000 \
  --max-model-len 262144 --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.85 --enforce-eager \
  --max-num-batched-tokens 16384

Target hardware: Blackwell (GB10 / sm_120+). The gridbook plugin JIT-builds its kernels on first load.

Honest limitations

No quality claims. There is no servable BF16 teacher for a 117B model on this hardware, so no KL/perplexity/benchmark deltas are published for this artifact. Gates that were run and passed: load, greedy-decoding coherence, 256k boot, 63k needle retrieval, packing/completeness checks, speed. Treat quality as unvalidated until an independent A/B exists.
The DFlash speculative-decoding drafter is not yet integrated/validated with this artifact; the numbers above are drafter-free.

Measured A/B vs poolside's NVFP4 release (same box, same protocol)

Identical protocol per arm: fresh boot, slack-gated util clearing the same 8 GiB floor (gridbook: 0.85 + --max-num-batched-tokens 16384, the serve command above; NVFP4: 0.85), vLLM eager, fp8 KV, greedy single-request timing.

	gridbook 6.0 bpp (this repo)	poolside NVFP4 (~4.5 bpp)
weights	84 GB	67 GB
decode	14.9 tok/s	19.1 tok/s
prefill @8k	2,186 tok/s (plugin ≥2026-07-28)	3,603 tok/s
prefill @63k	1,822 tok/s	2,403 tok/s
256k boot	yes	yes
63k needle	exact	exact

(gridbook numbers: plugin >= 2026-07-23 — the MoE prefill rides a CUDA chunk-expander into vLLM's own fused-MoE grouped kernel; serve with --max-num-batched-tokens 16384 and slack-gated util 0.85 as in the command above. Earlier plugin builds prefilled ~6x slower. Full-256k request concurrency at this config is 1.53x; drop --max-num-batched-tokens to trade prefill speed back for KV headroom.)

Reading it honestly: decode is per-byte neutral — the tok/s ratio equals the byte ratio (67/84 = 0.80), so gridbook's decode kernels move bytes exactly as fast as the native NVFP4 path; the deficit is purely that 6.0 bpp stores more bits per parameter than 4.5. Prefill trails by 1.65x at 8k — the remaining gap is the transient expand round-trip vs NVFP4's fused CUTLASS grouped GEMM, a known kernel roadmap item (persistent-N decode-in-mainloop), not a format property. No quality comparison is published for either arm (no servable BF16 teacher at this scale).

Attribution

Quantization by Robert Tand (robert.tand@icloud.com) — PrismaQuant / gridbook. Base model by poolside, served via vLLM.