Views
No views yet
[!NOTE] This repository contains official GGUF conversions from our standard format release built for llama.cpp (and compatible with vLLM and SGLang). To use with Ollama, pull directly withollama pull laguna-xs-2.1.llama.cpp support is not yet upstreamed. See below.
| File | Quant | Size |
|---|---|---|
Laguna-XS-2.1-BF16.gguf | BF16 (full precision) | 66.9 GB |
Laguna-XS-2.1-Q4_K_M.gguf | Q4_K_M | 20.3 GB |
Q4_K_M is the recommended default for local use. Use BF16 if you want a reference full-precision baseline or intend to produce your own quantizations.[!NOTE] Laguna XS 2.1 support is not yet in upstream llama.cpp. Until it lands, build llama.cpp from the PR that adds Laguna XS 2.1 support (ggml-org/llama.cpp#25165).
1# Build llama.cpp from the PR branch
2git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
3git fetch origin pull/25165/head:laguna && git checkout laguna
4cmake -B build && cmake --build build -j
5
6# Download a GGUF
7huggingface-cli download poolside/Laguna-XS-2.1-GGUF \
8 Laguna-XS-2.1-Q4_K_M.gguf --local-dir ~/models/Laguna-XS-2.1-GGUFllama-server:1./build/bin/llama-server \
2 -m ~/models/Laguna-XS-2.1-GGUF/Laguna-XS-2.1-Q4_K_M.gguf \
3 --jinja \
4 -ngl 99 \
5 -c 32768 \
6 --port 8000--jinja applies the model's built-in chat template (reasoning and tool-calling).-ngl 99 offloads all layers to the GPU (CUDA). Drop or lower it for CPU-only.-c sets the context length; the model supports up to 262,144 tokens, but a
bounded value (e.g. 32768) keeps KV-cache memory reasonable on local machines.[!NOTE] macOS (Metal) users: the same recipe works on Apple Silicon via the Metal backend. Enable flash attention (-fa on) for lower memory use and better throughput.