A CPU-offload-aware mixed-precision GGUF of Ornith-1.0-35B — deepreinforce-ai's RL-tuned reasoning model on the qwen35moe architecture. 256K context at ~78 tok/s decode on 18 GB of VRAM (vs ~61 tok/s for the uniform Q4_K_M), with perplexity within ~1% of the lossless Q8_0.

TL;DR: the strong-reasoner Ornith, running the full 256K window at small-context speed, in a 17 GB file.

What this is

Ornith-1.0-35B shares the hybrid qwen35moe architecture (gated attention + gated-delta-net SSM, 256 experts, 8+1 active, ~3B active params/token) but is RL-tuned for reasoning. This build applies the same mixed q2_K + imatrix quantization that the Qwen3.6 mixed quants use: the expert layers that get offloaded to system RAM are dropped to Q2_K (roughly half the bytes/token streamed from DDR4), while GPU-resident tensors stay at Q4_K. Decode is bound by RAM bandwidth on offloaded layers, so cutting those bytes is what buys the speed.

Recipe

Base: deepreinforce-ai/Ornith-1.0-35B (qwen35moe).
Source: the lossless Q8_0 GGUF, requantized with llama-quantize (--allow-requantize).
Importance matrix: computed on Ornith's own Q8_0 (~61K tokens / 119 chunks) so the Q2_K rounding is guided by this model's activation statistics — not borrowed from a different model.
Mixed layout (identical to the Qwen3.6 mixed quants):
- ffn_*_exps on blocks 13–26 → Q2_K (CPU-offloaded set, 42 tensors)
- everything else → Q4_K (GPU-resident) · output-class → Q6_K
4.88 bpw effective, ~17 GB, 40 blocks, 256K native context.

See the Gordon (Qwen3.6 base) mixed quant card for the full explanation of the offload-byte-count rationale.

Benchmarks

RTX 3060 Ti (8 GB) + RTX 3080 (10 GB), Ryzen 5950X, 46 GB DDR4-2733, ik_llama.cpp, q4_0 KV, flash-attn on:

Metric	This mixed q2_K	Ornith Q4_K_M	Ornith Q8_0
Decode	~78 tok/s @256K	~61 tok/s @256K	~32 tok/s @64K
Context	262144	262144	65536
VRAM	~16.5 GB	~15.4 GB	~15.6 GB (+~26 GB RAM)
Tool-calls	clean Qwen `tool_calls`	clean	clean
Perplexity (holdout, ctx 512)	2.3676	—	2.3434

Perplexity A/B vs the Q8_0 reference: +1.03% (near the ±0.06 confidence interval). Ornith's RL-tuned weights are slightly more sensitive to the Q2_K layers than the base Qwen3.6 (which is +0.59%), but the loss is still small for a 2.4× decode speedup and 4× context.

How to run (ik_llama.cpp)

The -ot override is required — it pins the Q2_K layers (13–26) to CPU so the low-bit tensors are exactly the offloaded ones. Do not add --n-cpu-moe with this build.

bash

./llama-server \
  -m ornith-1.0-35b-mixed-q2k.gguf \
  --jinja --cache-type-k q4_0 --cache-type-v q4_0 --flash-attn on \
  --ctx-size 262144 --parallel 1 --n-gpu-layers 99 \
  -ot 'blk\.(1[3-9]|2[0-6])\.ffn_(up|gate|down)_exps\.weight=CPU' \
  --tensor-split 44,56 --ubatch-size 256 --no-mmap --threads 8 --no-warmup \
  --port 8000

Ornith is a reasoning model — it emits a chain of thought before answering. Keep reasoning enabled for hard problems; it produces clean Qwen tool_calls for agent use.

Intended use & limitations

Target: local long-context reasoning on ~16–18 GB VRAM at usable speed.
The Q2_K expert layers are the quality floor; this model loses a touch more than the Qwen3.6 mixed quant (~1% PPL). For maximum fidelity use the Q8_0.
Inherits the capabilities and biases of the base Ornith-1.0-35B. Pure quantization — no fine-tuning or alignment changes.

Provenance

Base model: Ornith-1.0-35B by deepreinforce-ai (MIT).
Mixed quantization + imatrix + tuning: xero0000, June 2026.

Released under the base model's MIT license.

xero0000/Vortigaunt-35B-mixed-q2k-imat

Vortigaunt — Ornith-1.0-35B Mixed q2_K + imatrix