A CPU-offload-aware mixed-precision GGUF of Qwen-AgentWorld-35B-A3B — the official Qwen "language world model", trained to simulate agent environments (MCP/tool-calling, Search, Terminal, SWE, Android, Web, OS). Runs the full 256K context at ~79 tok/s decode on a single 18 GB dual-GPU desktop, by quantizing the CPU-offloaded expert layers to Q2_K while keeping the GPU-resident tensors at Q4_K.

TL;DR: an ~18 GB file that runs a Claude-Sonnet-class agentic model (AgentWorldBench 56.4) at full 256K context, at small-context speed — the daily driver for browser/tool/OS agent loops.

What it is

Qwen-AgentWorld-35B-A3B is Qwen's agent-environment world model on the hybrid qwen35moe architecture (gated attention

gated-delta-net SSM, 256 experts, 8+1 active, ~3B active/token). It is thinking-ON by design — it reasons about environment state in <think> before acting — and scores 56.4 on AgentWorldBench, roughly Claude-Sonnet-class on agentic tasks.

This build applies the mixed q2_K + imatrix recipe so you get that model at the fast 256K profile on limited VRAM. GGUF lineage: quantized from DevQuasar's Q8_0 conversion (the original safetensors use Qwen3_5MoeForConditionalGeneration, which llama.cpp's converter doesn't handle directly).

Recipe

Mixed layout:
- ffn_*_exps on blocks 13–26 → Q2_K with importance matrix (the CPU-offloaded set)
- everything else → Q4_K · output-class → Q6_K
~4.9 bpw effective, ~18 GB, 256K native context.
Why it works: decode on CPU-offload rigs is RAM-bandwidth-bound, so only the offloaded-layer bytes matter — those get Q2_K while GPU-resident tensors keep Q4_K quality. See the Gordon card for the full rationale.

Measured performance

Rig: RTX 3060 Ti 8 GB + RTX 3080 10 GB, DDR4, ik_llama.cpp:

profile	ctx	decode
this file, `-ot` CPU-offload	262144	~79 tok/s
DevQuasar Q8_0, ncmoe 26	65536	~32 tok/s

Want more speed? Alyx-MTP is this same file with a transplanted multi-token-prediction head for self-speculative decoding in ik_llama.cpp (~79-82 tok/s at 128K).

How to run

Works in llama.cpp and ik_llama.cpp. The tuned 18 GB dual-GPU profile (ik):

bash

./llama-server -m Qwen-AgentWorld-35B-A3B-mixed-q2k.gguf \
  --jinja --cache-type-k q4_0 --cache-type-v q4_0 --flash-attn on \
  --ctx-size 262144 --parallel 1 --n-gpu-layers 99 --ctx-checkpoints 8 \
  -ot 'blk\.(1[3-9]|2[0-6])\.ffn_(up|gate|down)_exps\.weight=CPU' \
  --tensor-split 44,56 --ubatch-size 256 \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  --no-mmap --threads 8 --no-warmup

Do NOT disable reasoning — the <think> environment-state reasoning is the point of this model. Give it generous max_tokens.
Recommended sampling (per the base model): temp 0.6, top-p 0.95, top-k 20.
--ctx-checkpoints 8 caps ik's dynamic SSM checkpoints (default 32 × 64 MiB ≈ 2 GB at deep context — an OOM trap on long agent sessions).

Credits

Base model: Qwen/Qwen-AgentWorld-35B-A3B (Apache-2.0)
Q8_0 GGUF source: DevQuasar
Mixed quant, imatrix & profiling: xero0000
Series: Gordon (base) · Kleiner (coder) · G-Man (uncensored) · Vortigaunt (reasoner) · Alyx (agentic)

xero0000/Alyx-AgentWorld-35B-A3B-mixed-q2k

🤝 Alyx — Qwen-AgentWorld-35B-A3B Mixed q2_K + imatrix

What it is

Recipe

Measured performance

How to run

Credits

xero0000/
Alyx-AgentWorld-35B-A3B-mixed-q2k