You can deploy this model and start earning money today!

LiamVisionary/
swarm-sovereign-scout-12b-GGUF

Name: swarm-sovereign-scout-12b-GGUF
Availability: InStock

Views

No views yet

Swarm Scout 12B GGUF

Swarm Scout 12B is a low-refusal, coding/agentic GGUF build derived from yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2, itself based on google/gemma-4-12B-it.

The upstream model is specialized for coding, terminal workflows, native Gemma 4 tool-use, multi-step agentic tasks, and reasoning. This release keeps that focus, then applies a local Heretic/abliteration pass targeted at reducing opening refusals while preserving behavior as much as possible.

Brand note: this release is now named Swarm Scout. The Hugging Face repo slug and GGUF filenames still contain swarm-sovereign-scout so existing download commands, Modal caches, and copied links keep working.

Build With HivemindOS

Want to build and manage an entire private swarm of agents? HivemindOS gives you shared memory, reusable skills, one-time setup, and a superbrain architecture for coordinating agent work.

App: HivemindOS
X: @TheHivemindOS

Artifacts

File	Size	Purpose
`swarm-sovereign-scout-Q4_K_M.gguf`	~6.9 GB	Recommended local/runtime artifact
`MTP/gemma-4-12B-it-MTP-Q8_0.gguf`	~444 MB	Draft model for llama.cpp MTP speculative decoding
`mmproj-swarm-sovereign-scout-bf16.gguf`	~167 MB	Projector artifact for llama.cpp multimodal image input
`chat_template.jinja`	small	Swarm Scout identity/tool-use template override for llama.cpp

The local workspace also produced a BF16 GGUF (swarm-sovereign-scout-bf16.gguf, ~22 GB). Q4_K_M is the recommended release/runtime target for speed and practicality.

What It Is Good At

Coding and debugging assistance.
Terminal-style agent tasks: read, reason, act, verify.
Structured tool-use in Gemma 4's native protocol.
Local agent loops where lower refusal tendency is desired.
Reasoning-heavy technical work, especially with thinking enabled.

The upstream base reports a large relative gain on local tau2-bench telecom-style agentic tool-use versus google/gemma-4-12B-it; see the upstream model card for the exact methodology and caveats.

Abliteration / Refusal Check

Local verified artifact:

Winning Heretic trial: 20
Heretic proxy refusals: 0/6
KL divergence: 0.008679866790771484
Merged HF strict refusal check: 0/6 strict full-response refusals
Served Q4_K_M llama.cpp strict refusal check: 0/6 strict full-response refusals

The strict check measures opening/full-response refusal behavior on a small local prompt set. It is not a broad safety benchmark and should not be treated as a guarantee of behavior in production. Add application-level guardrails where needed.

Speculative Decoding

This release includes the Gemma 4 MTP draft model used for DSpark-style speculative decoding in llama.cpp.

Verified local build:

llama.cpp b9553, commit 9e3b928fd8c9d14dbf15a8768b9fdd7e5c721d66
Draft type: draft-mtp
Recommended Scout settings: --spec-draft-n-min 0 --spec-draft-n-max 2
Draft file: MTP/gemma-4-12B-it-MTP-Q8_0.gguf

Verified Modal build:

llama.cpp image: ghcr.io/ggml-org/llama.cpp:server-cuda-b9870
GPU: Modal A10
Server command uses -md MTP/gemma-4-12B-it-MTP-Q8_0.gguf --spec-type draft-mtp
Production smoke returned draft_n / draft_n_accepted, confirming draft tokens were active.
Hosted endpoint auth: Modal proxy auth is enabled. Server-side callers must send Modal proxy headers; do not expose those credentials in browser/mobile clients.

Older llama.cpp builds may not understand the Gemma 4 assistant draft architecture. For example, server-cuda-b9544 sees CUDA but fails to load this draft with unknown model architecture: 'gemma4-assistant'.

Clean local Apple Silicon measurements with reasoning disabled:

Prompt set / config	Baseline tok/s	MTP tok/s	Speedup	Acceptance
Mixed, `n=0..2`	`48.12`	`70.41`	`1.46x`	`0.75163`
Code, `n=0..2`	`33.96`	`47.93`	`1.41x`	`0.69159`
Mixed, `n=1..3`, `p_min=0.05`	`37.54`	`60.14`	`1.60x`	`0.63131`

Use absolute tokens/sec as the tuning objective. The 1.60x row exists, but n=0..2 produced better real throughput on the local machine.

llama.cpp Usage

Download the model and draft:

bash

hf download LiamVisionary/swarm-sovereign-scout-12b-GGUF \
  swarm-sovereign-scout-Q4_K_M.gguf \
  MTP/gemma-4-12B-it-MTP-Q8_0.gguf \
  mmproj-swarm-sovereign-scout-bf16.gguf \
  chat_template.jinja \
  --local-dir swarm-scout-12b

Run an OpenAI-compatible server:

bash

llama-server \
  -m swarm-scout-12b/swarm-sovereign-scout-Q4_K_M.gguf \
  -md swarm-scout-12b/MTP/gemma-4-12B-it-MTP-Q8_0.gguf \
  --mmproj swarm-scout-12b/mmproj-swarm-sovereign-scout-bf16.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-min 0 \
  --spec-draft-n-max 2 \
  -ngl 99 \
  -ngld 99 \
  -fa on \
  --jinja \
  --chat-template-file swarm-scout-12b/chat_template.jinja \
  --reasoning auto \
  --reasoning-format deepseek \
  --alias swarm-scout-12b \
  --host 127.0.0.1 \
  --port 8080

Then call:

bash

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "swarm-scout-12b",
    "messages": [{"role": "user", "content": "Write a small Python LRU cache."}],
    "max_tokens": 512,
    "temperature": 0
  }'

Image input uses OpenAI-compatible content parts:

json

{
  "model": "swarm-scout-12b",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Read this image and summarize it."},
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
      ]
    }
  ],
  "max_tokens": 256,
  "temperature": 0
}

Reasoning

For fast agent loops, run with reasoning off:

--reasoning off --reasoning-format none

For harder debugging/planning, use:

--reasoning auto --reasoning-format deepseek

Compatible clients may also pass thinking_budget_tokens and chat_template_kwargs.enable_thinking per request.

Multimodal Status

The underlying Gemma 4 unified configuration includes image/video/audio processor metadata. The GGUF text model requires the matching mmproj-swarm-sovereign-scout-bf16.gguf projector for multimodal serving in llama.cpp.

Verified on 2026-07-05:

Local llama.cpp b9553 loaded swarm-sovereign-scout-Q4_K_M.gguf plus mmproj-swarm-sovereign-scout-bf16.gguf.
Modal llama.cpp server-cuda-b9870 advertised capabilities: ["completion", "multimodal"].
A receipt image test returned Merchant: SWARM CAFE. Total: $8.37. through the production OpenAI-compatible endpoint.

Image input is verified. The projector metadata also exposes an audio encoder, but audio/video input was not separately quality-tested for this release.

Caveats

Specialized for coding, terminal, tool-use, and local agentic work.
General-knowledge answers should be checked.
Low-refusal behavior is intentional; production applications should add their own policy and abuse-prevention layers.
English-centric.
Speculative decoding support is build-sensitive. The local verified path used llama.cpp b9553; the Modal verified path used server-cuda-b9870; newer/older builds may need their own validation.

Lineage

Base model: google/gemma-4-12B-it
Agentic/coding base: yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2
GGUF/MTP reference repo: yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF

Released under Apache 2.0, following the upstream/base lineage.