A mixture-of-experts, multimodal (vision-language) model with a Hermes-style agentic / function-calling alignment, distributed as a compact GGUF for llama.cpp / llama-server.

In the author's local 8-model benchmark suite, this build placed #1 in every category — math, knowledge, and agentic tool-use — making it the top composite scorer of the set.

Benchmark results — Qwen3.6-35B-Hermes-V6 ranks #1 across math, knowledge, and agentic tasks

📊 The full interactive report is also bundled in this repo: FINAL_REPORT.html.

✨ What this is


Base lineage	Qwen 3.6 family (community build / fine-tune — not an official Alibaba release)
Architecture	Mixture-of-Experts (MoE), Qwen3
Total / active params	~35B total / ~3B active per token (the `A3B` notation)
Modality	Text + Vision (ships with a separate `mmproj` multimodal projector)
Alignment	Hermes V6 — agentic, function-calling, instruction-tuned
Format	`GGUF` — `APEX-Compact` mixed-precision (~4 bpw-class)
Main file size	~17 GB
Serving	`llama.cpp` / `llama-server` (OpenAI-compatible API)

The MoE design is the key to its efficiency: although ~35B parameters are loaded, only ~3B are active on any given token, so it runs at usable speeds on consumer APUs/GPUs while keeping dense-model quality.

📦 Files in this repository

File	Size	Description
`Qwen3.6-35B-A3B-Hermes-V6-APEX-Compact.gguf`	~17 GB	Main model weights (APEX-Compact GGUF). Load this with `llama-server -m`.
`Qwen3.6-35B-A3B-Hermes-V6-mmproj.gguf`	~858 MB	Multimodal projector (vision encoder adapter). Load with `--mmproj`. Required for image input.
`FINAL_REPORT.html`	~13 KB	Interactive benchmark report (champion board + full breakdown).

🏆 Benchmark results

Measured by the author on local hardware (see Methodology & caveats below). This model is 🥇 #1 in the 8-model comparison.

Composite champion board (top of the set)

Model	Size	GSM8K (math)	Knowledge	Agentic	Composite	Speed
🥇 Qwen3.6-35B-Hermes-V6 (this)	16.2 GB	91.0%	78.1%	90.0%	86.4%	~15 t/s
🥈 Gemma-4-31B	8.0 GB	84.0%	73.4%	90.0%	82.5%	6.5 t/s
🥉 Gemma-4-12B-QAT	6.3 GB	88.5%	75.2%	80.0%	81.2%	10.6 t/s

Knowledge benchmarks — full breakdown (200 questions each, 2 runs averaged)

Benchmark	Score
ARC-Challenge	93.0%
HellaSwag	85.5%
OpenBookQA	94.5%
BoolQ	89.5%
Winogrande	75.0%
TruthfulQA	31.0%
Knowledge average	78.1%

Math & agentic

GSM8K: 91.0% — via lm-evaluation-harness (industry standard).
Agentic: 90.0% (9/10) — τ-bench-style retail customer-service simulator (multi-turn, strict pass/fail): cancel orders, returns, refunds, exchanges, policy refusals, error recovery, and tool restraint.

⚠️ TruthfulQA is low for every model in this set (25–31%). TruthfulQA is adversarial by design; these subsampled, locally-judged scores are not directly comparable to official leaderboard figures.

🚀 How to run

Prerequisites

A recent build of llama.cpp (CPU, Vulkan, or ROCm/CUDA backend).
Enough RAM/VRAM for ~17 GB of weights (the MoE means compute is light, but all experts must reside in memory).

Serve it (OpenAI-compatible API, with vision)

bash

llama-server \
  -m Qwen3.6-35B-A3B-Hermes-V6-APEX-Compact.gguf \
  --mmproj Qwen3.6-35B-A3B-Hermes-V6-mmproj.gguf \
  -ngl 99 \
  -c 32768 \
  --port 8080 \
  --host 0.0.0.0

Then talk to it like any OpenAI API at http://localhost:8080/v1/chat/completions.

Function calling / tool use

The Hermes V6 alignment supports the standard OpenAI tools schema (tool_choice: auto). Example:

bash

curl -s http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "Qwen3.6-35B-A3B-Hermes-V6-APEX-Compact.gguf",
  "messages": [{"role":"user","content":"Find my pending order and cancel only that one."}],
  "tools": [ /* your function schemas */ ],
  "tool_choice": "auto",
  "temperature": 0.1
}'

Thinking mode control

This is a hybrid reasoning model. Prefix the system prompt with /no_think for fast, non-reasoning responses (recommended for high-volume tool-calling / agent loops); omit it to engage chain-of-thought reasoning.

Multimodal (image) input

With the --mmproj projector loaded, you can pass images (via the llama-server multimodal interface) alongside text for image-understanding tasks.

🔬 Methodology & caveats (please read)

These results are author-measured on a specific local setup, not official leaderboard scores. Be aware:

Knowledge (MCQ): 200-question subsamples per dataset, averaged over 2 independent runs (matched within ±2.5%).
GSM8K: run through lm-evaluation-harness (EleutherAI) — the industry-standard harness.
Agentic: a custom τ-bench-style simulator (10 multi-turn retail tasks, strict pass/fail). The headline figure is the best of dedicated + loop runs, each model run with its optimal flags.
Hardware: AMD Ryzen AI 9 HX 470 · Radeon 890M (~14.4 GB) · 28 GB RAM · Vulkan/ROCm backend. Speeds (~15 t/s) are specific to this device.
Subsampled/local judging means absolute numbers are illustrative, not directly comparable to full-scale published benchmarks. Use them to rank models within this set, not to compare against vendor-published scores.

Full details and the interactive charts are in FINAL_REPORT.html.

⚖️ Licensing & attribution

This is a community release by MrFadiAi. It is not an official Alibaba / Qwen or Nous Research / Hermes product.
The license: apache-2.0 field follows the upstream Qwen3 lineage. Verify and comply with the applicable upstream Qwen license for your use case before redistribution or commercial deployment.
"Hermes" and "Qwen" are trademarks of their respective owners and are used here only to describe the model's training lineage.

🙏 Acknowledgements

Built on the outstanding work of the Qwen team, the Hermes alignment tradition, the llama.cpp project, and EleutherAI's lm-evaluation-harness.

MrFadiAi/Qwen3.6-35B-A3B-Hermes-V6

Qwen3.6-35B-A3B-Hermes-V6 (APEX-Compact, GGUF)