A 4-bit MXFP4 quantization of poolside/Laguna-S-2.1, produced with
qstream. The routed MoE experts (~96% of the
weights) are quantized to MXFP4; everything quality-sensitive stays BF16.
sensitive / runs on every token — kept lossless from the source
Embeddings, lm_head, norms
BF16
unchanged
Quality & faithfulness
We lead with deterministic, reproducible faithfulness metrics. (Task evals served
over vLLM with continuous batching are not bitwise-deterministic — the same prompt at
temperature=0 can yield different reductions depending on batch composition — so
task accuracies carry a little serving noise; we quote one full-test-set score, where
that noise is small.)
Metric
Result
What it shows
Perplexity (wikitext-2 test, 309k tokens)
9.82
language modeling intact — a broken quant lands in the hundreds
Routed-expert SQNR
≈ 19 dB
reconstruction error is just the unavoidable 4-bit rounding (MXFP4 vs the BF16 source)
GSM8K (full test, 1319 questions, temp 0)
94.5%
end-to-end reasoning intact through the quantized experts
Why this is enough to trust the checkpoint:
Only the routed experts changed. ~96% of the weights are re-quantized to MXFP4;
everything else is bit-identical BF16 to the source. So the model is
the base model except for 4-bit rounding on the expert GEMMs.
The error is uniform and diffuse. All 36,096 expert tensors sit in a tight band
(median relative error 0.1117, p95 0.1124, max 0.1356) with no pathological layers —
pure 4-bit rounding, no outliers to hide.
PPL script: evals/eval_ppl_wikitext2.py in the qstream repo.
Serving with vLLM
Laguna is natively supported in vLLM ≥ 0.23, and this checkpoint uses the stock
compressed-tensors MXFP4 path — no patches needed:
Keep --max-model-len bounded: only the 12 full-attention layers grow KV with
context (~48 KiB/token in BF16 KV), the other 36 layers hold a fixed 512-token
sliding window. The native 1M context needs ~46 GiB of KV per sequence.
Observed throughput: ≈600 tok/s aggregate generation at batch size 16 on
a single RTX PRO 6000 Blackwell (96 GB) — the whole checkpoint plus KV fits
on one card.
FP8 KV cache — works uncalibrated
--kv-cache-dtype fp8 is a verified free lunch on this model: no calibration
or static k/v scales needed. Measured against BF16 KV: wikitext-2 PPL 9.812 vs
9.816, passkey retrieval 20/20 vs 20/20 at all depths in ~30k-token contexts.
(Laguna's per-head k_norm keeps keys inside e4m3 range at scale 1.0.) This
halves KV to ~24 KiB/token — double the context or concurrency.
On SM120 (RTX PRO 6000 / Blackwell workstation) three extra knobs are required:
FlashInfer because FA2 can't read fp8 KV; disable_flashinfer_q_quantization
because vLLM otherwise also quantizes queries to fp8, which flashinfer's fa2
prefill template rejects outside the SM100 trtllm-gen path; the 1 GiB workspace
because the default 394 MiB overflows on Laguna's 72-head sliding-window layers.
Activation-aware (γ-weighted) MSE scale selection with per-block search over
candidate E8M0 exponents; the BF16 remainder is passed through untouched and the
mixed-precision config.json (ignore list + expert targets) is emitted for
stock vLLM loading. Built with qstream@9f8cd0c.
License
Inherits the OpenMDW-1.1 license from the base model. This is a
derivative (quantized) work of poolside's Laguna S 2.1.
Laguna S 2.1 is a 118B total parameter Mixture-of-Experts model with 8B activated
parameters per token, designed for agentic coding and long-horizon work. It sits
between Laguna XS 2.1 (33B-A3B) and
Laguna M.1 (225B-A23B) in the Laguna series and shares the family recipe: a
token-choice router with softplus gating over 256 routed experts plus one shared
expert, grouped-query attention, and interleaved full/sliding-window attention.
Highlights
Mixed SWA and global attention layout: 48 layers in a 1:3 global-to-SWA ratio
(12 global attention layers, 36 sliding-window layers, window 512), with softplus
attention gating and per-layer-type rotary scales
1M context: 1,048,576-token context window
Native reasoning support: interleaved thinking between tool calls, with
per-request control via enable_thinking
Speculative decoding: a trained
DFlash draft model is available
for lower-latency serving
OpenMDW-1.1 license: Use and modify the model and associated materials freely
for commercial and non-commercial purposes
(learn more about OpenMDW)
Model overview
Number of parameters: 118B total, ~8B activated per token
Layers: 48 (12 global attention, 36 sliding-window attention)
Experts: 256 routed (top-10) plus 1 shared expert
Attention: grouped-query, 8 KV heads, head dim 128; per-head softplus output gating
Sliding window: 512 tokens
Context window: 1,048,576 tokens
Vocabulary: 100,352 tokens (Laguna family tokenizer)
Modality: text-to-text
Reasoning: interleaved thinking with preserved thinking
Benchmark results
benchmarks
Model
Size
Terminal-Bench 2.1
SWE-bench Multilingual
SWE-Bench Pro (Public Dataset)
DeepSWE
SWE Atlas (Codebase QnA)
Toolathlon Verified
Laguna S 2.1
118B-A8B
70.2%
78.5%
59.4%
40.4%
46.2%
49.7%
Tencent Hy3
295B-A21B
71.7%
75.8%
57.9%
-
-
-
Inkling
975B-A41B
63.8%
-
54.3%
-
-
45.5%*
Nemotron 3 Ultra
550B-A55B
56.4%
67.7%
-
-
-
34.3%*
DeepSeek-V4-Pro Max
1.6T-A49B
64.0%*
76.2%
55.4%
9.0%*
27.2%*
55.9%*
Kimi K3
2800B-A50B
88.3%
-
-
69%
-
-
Qwen 3.7 Max
-
74.5%*
78.3%
60.6%
-
-
-
Muse Spark 1.1
-
80%
-
61.5%
53.3%
42.2%*
75.6%
Claude Fable 5
-
88%
-
80.3%
70%
-
-
Benchmarks as of 21 July 2026. Laguna S 2.1 in bold; a dash (-) marks a benchmark a model was not evaluated on. Scores marked * are as reported by third parties: Terminal-Bench 2.1 and DeepSWE via Artificial Analysis, SWE Atlas via Scale AI's official leaderboard, and Toolathlon Verified via its official leaderboard. Full evaluation trajectories: trajectories.poolside.ai.
Usage
Laguna S 2.1 uses the same laguna architecture as Laguna XS 2.1, so the same
engine integrations apply (vLLM, SGLang, Transformers, TRT-LLM, llama.cpp). At 118B
parameters the BF16 checkpoint needs multiple GPUs (roughly 236GB of weights);
quantized variants reduce this substantially.
[!NOTE]
Optional: speculative decoding with DFlash. Pair with the
Laguna S 2.1 DFlash draft model
by adding
--speculative-config '{"model":"poolside/Laguna-S-2.1-DFlash","num_speculative_tokens":7,"method":"dflash"}'.
Note the flag names differ from vLLM's (--tool_parser, and the reasoning parser
is laguna, not poolside_v1).
llama.cpp
GGUF conversions are available at
poolside/Laguna-S-2.1-GGUF.
Serve with poolside's llama.cpp fork, branch
laguna, which carries
full Laguna support including DFlash speculative decoding. (Base Laguna support
is also in upstream review:
ggml-org/llama.cpp#25165.)
Laguna S 2.1 has native reasoning support and works best with preserved thinking:
keep reasoning_content from prior assistant messages in the message history.
The model will generally reason before calling tools and between tool calls, and
may stop reasoning in follow-up steps if prior thinking blocks are dropped.
Thinking is controlled per request via the chat template:
or at the server level with
--default-chat-template-kwargs '{"enable_thinking": true}'. For agentic coding
use cases we recommend enabling thinking and preserving reasoning in the message
history.
Laguna S 2.1 is designed for software engineering and agentic coding use cases, and you are responsible for confirming that it is appropriate for your intended application. Laguna S 2.1 is subject to the OpenMDW-1.1 License, and should be used consistently with Poolside's Acceptable Use Policy. We advise against circumventing Laguna S 2.1 safety guardrails without implementing substantially equivalent mitigations appropriate for your use case.
Please report security vulnerabilities or safety concerns to security@poolside.ai.
