Laguna S 2.1 Hybrid 2.75bpw
Verified public package. The weight transform, TP2 runtime, parser,
structured-output, 195k-context, throughput, package-manifest, and Hub-object
verification gates completed before public release.
Laguna S 2.1 Hybrid 2.75bpw is a two-tier, expert-level quantization of
poolside/Laguna-S-2.1-NVFP4.
It preserves every expert and the source router topology. REAP saliency is used
only to choose the NVFP4 hot tier; it is not expert pruning.
Quantization layout
Each of the 47 sparse MoE layers has 256 routed experts:
| Tier | Experts per sparse layer | Storage |
|---|
| Hot | 96 | source NVFP4, retained byte-exact |
| Tail | 160 | calibrated EXL3 Trellis-2 |
This is a nominal 2.75-bit mixture across the expert weights. Shared experts,
the router, attention, embeddings, LM head, and other non-expert tensors remain
in their source representation. All 256 experts remain routable.
The packaged source shards are compacted: they contain only the 96 selected
NVFP4 routed experts per sparse layer. The other 160 routed experts are present
only in the calibrated Trellis-2 tail files, so the package does not duplicate
their original NVFP4 tensors. The release manifest seals 50,111,846,955 bytes
(approximately 46.67 GiB), excluding the manifest file itself. The 2.75-bpw
label describes routed-expert weight storage, not the whole repository after
non-expert tensors and metadata.
How the hot tier was selected
The hot experts are ranked independently in each sparse layer using router-
weighted expert-output activation saliency over a held-out calibration mixture
covering coding, tool and agentic calls, reasoning, mathematics, science,
terminal work, CUDA, cybersecurity, and long-context prompts. The score is the
mean, over tokens routed to an expert, of the applied router weight multiplied
by that expert's output L2 norm. The release includes the exact tier map, source
revision, calibration provenance, and artifact hashes.
Runtime and validation
The hybrid requires the accompanying EXL3-tail runtime and is not a drop-in
stock-vLLM checkpoint. The runtime supports single-Spark TP1 by reconstructing
both stored tail slices on one device, and two-Spark TP2 by loading one stored
slice per rank. The package includes a reproducible Dockerfile, the pinned EXL3
source archive and ARM patch, the runtime overlay, and the exact launch recipes.
- Tested ARM64 base-runtime source:
Anemll/dspark-vllm-gx10
- Published base image:
ghcr.io/anemll/dspark-vllm-gx10:0.1.1
- Exact Laguna hybrid overlay:
runtime/Dockerfile
The sealed validation bundle records:
- all 47 sparse layers loaded as 96 NVFP4 + 160 Trellis-2 experts in both the
compact TP1 and distributed TP2 layouts;
- normal CUDA graph capture, with no eager execution or graph disabling;
- 200,000-token configured context and a valid approximately 195k-token request;
- separately parsed reasoning, exact tool calling, tool history, strict JSON
schema output, and byte-exact ASCII output;
- two-Spark TP2 direct-generation throughput of 24.13, 49.62, and 77.89
completion tokens/s at concurrency 1, 2, and 4, respectively.
The source checkpoint has no native MTP tensors. The pinned Laguna DFlash draft
model was tested but rejected for this compressed hybrid after preserved
sustained-decode divergence/nontermination evidence. The verified default is
therefore LAGUNA_SPECULATIVE_MODE=none; DFlash remains an explicit
experimental opt-in and is not claimed as validated.
No Terminal-Bench 2.1 quality score is claimed in this card.
Provenance
- Base model:
poolside/Laguna-S-2.1-NVFP4
- Base-model license: OpenMDW-1.1
- Hybrid method: source NVFP4 hot experts plus calibrated EXL3 Trellis-2 tails
- REAP reference: REAP the Experts
Responsible use
This derivative remains subject to the upstream OpenMDW-1.1 license and should
be used consistently with the upstream model's acceptable-use guidance.
