A 122B hybrid Gated-DeltaNet MoE (256 experts, 8 active) — packed to 48 GiB so it runs on one 80 GB GPU (or a small card + CPU offload). Quantized from the original BF16 with an importance matrix (math + code + general calibration), via llama.cpp.

Framework, recipe, and full reproduction: https://github.com/General-Instinct/InstinctRazor

Speed (llama.cpp, this artifact)

1× H100-80GB, all layers on GPU: 115.9 tok/s decode (prefill ≈2541 tok/s).
Small card + CPU expert-offload (--n-cpu-moe 48, peak ≈7.6 GiB VRAM): 45.7 tok/s decode — runs on an 8 GB GPU + ≈48 GiB system RAM.

Run

bash

# full GPU
llama-cli -m InstinctRazor-Qwen3.5-122B-A10B-IQ3_XXS.gguf -ngl 999 -fa on -p "Your prompt"
# small card + CPU offload (routed experts on CPU)
llama-cli -m InstinctRazor-Qwen3.5-122B-A10B-IQ3_XXS.gguf -ngl 999 --n-cpu-moe 48 -t 52 -p "Your prompt"
# multimodal (image input)
llama-cli -m InstinctRazor-Qwen3.5-122B-A10B-IQ3_XXS.gguf --mmproj InstinctRazor-Qwen3.5-122B-A10B-mmproj-f16.gguf --image pic.png -p "Describe the image"

Requires a llama.cpp build with qwen3_5_moe support (upstream, 2026-02+).

Scope & roadmap

This GGUF matches or beats the footprint-matched A4B on knowledge, reasoning, and multimodal-MMMU. Where it still trails — code (LiveCodeBench v6) and math / multimodal-math — the loss is largely token-inefficiency introduced by quantization, and is the target of OPD (on-policy distillation), a separate framework we'll open-source later. Eval absolutes are subject to a same-harness validation gate; see the GitHub results/RESULTS.md for full per-number provenance.

Attribution

Quantization recipe + framework: General Instinct, released under Apache-2.0.

General-Instinct/InstinctRazor-Qwen3.5-122B-A10B-GGUF

InstinctRazor — Qwen3.5-122B-A10B · IQ3_XXS GGUF

Speed (llama.cpp, this artifact)

Run

Scope & roadmap

Attribution

General-Instinct/
InstinctRazor-Qwen3.5-122B-A10B-GGUF