GGUF + importance-matrix quantizations of Nex-N2-mini — the Qwen3.5-35B-A3B MoE (~3B active, multimodal reasoning) by the Nex-AGI team — tuned and measured for fast local inference on the Intel Arc Pro B70 (Battlemage) via llama.cpp's SYCL backend.

The model is Nex-N2-mini, built by the Nex-AGI team — the reasoning, multimodal capability, and training are entirely theirs; this is an independent community quantization that only repackages their model to run fast on Intel Arc, with sincere thanks to Nex-AGI for releasing it openly. The base model is © Nex-AGI (Apache-2.0), post-trained on Qwen3.5-35B-A3B-Base © Qwen Team (Apache-2.0); these quants are redistributed under Apache-2.0 and the names identify the upstream work.

Code, kernel patch, full methodology, and reproducible benchmarks: https://github.com/newjordan/NexN2-B70-Turbo

Files

file	quant	size	bpw	mean KLD ↓ vs Q6_K	top-1	decode t/s (B70)
`Nex-N2-mini-B70-Turbo-Q5_K_M.gguf` ⭐	Q5_K_M	23.0 GB	5.71	0.0201	94.0%	88.1
`Nex-N2-mini-B70-Turbo-Q4_K_M.gguf` ⚡	Q4_K_M	19.7 GB	4.88	0.0389	91.6%	93.5
`Nex-N2-mini-B70-Turbo-Q4_K_dyn.gguf`	Q4_K_dyn	21.3 GB	5.27	0.0277	93.1%	78.3
`Nex-N2-mini-B70-Turbo-IQ4_XS.gguf`	IQ4_XS	17.4 GB	4.32	0.0466	90.8%	52.8
`Nex-N2-mini-B70-Turbo-Q3_K_dyn.gguf`	Q3_K_dyn	17.1 GB	4.24	0.0848	88.0%	64.6
`Nex-N2-mini-B70-Turbo-Q3_K_M.gguf`	Q3_K_M	15.6 GB	3.87	0.1048	86.3%	62.1
`Nex-N2-mini-B70-Turbo-Q3_K_S.gguf`	Q3_K_S	14.1 GB	3.50	0.1479	83.9%	52.2
`mmproj-f16.gguf`	vision projector	0.8 GB	—	—	—	—

Recommendation: Q5_K_M (best accuracy under the Q6_K reference, near-fastest) or Q4_K_M for max speed. On SYCL, Q4_K / Q5_K carry the optimized kernels and lead the decode column — they're the speed picks. Accuracy is KL-divergence + top-1 agreement vs a Q6_K reference (PPL 6.572, wikitext-2, 100 chunks). For vision, add mmproj-f16.gguf.

Benchmarks (Intel Arc Pro B70, real measurements)

With the project's Turbo package on Arc Pro B70:

config	decode @ ctx0	decode @ 131k
fresh stock control: reorder off / FA off	68.8 t/s	20.0 t/s
NexN2 B70 Turbo Q5_K_M (accuracy pick; MoE reorder + FA + retained NX2 fusions)	88.1 t/s	42.3 t/s
NexN2 B70 Turbo Q4_K_M (speed pick; MoE reorder + retained NX2 fusions)	93.5 t/s	not remeasured

Q5_K_M/Q4_K_M decode updated 2026-06-12 with the Turbo patch chain's Q6_K MoE reorder (patches/0003 in the GitHub repo). patches/0004 adds default-on exact NX2 fused gate/up SwiGLU, post-down weighted-sum fusion, and fused F32 MoE tail add (Q5_K_M ctx0 reaches 88.1 t/s in the retained default; Q4_K_M reaches 93.5 t/s on the release-card evidence). The 131k Q5_K_M table entry is a full-package measurement, not a fused-MoE-only attribution: the audited release gate measured 0004 at 42.0952 -> 42.3148 t/s (+0.52%) for Q5_K_M 131k. The large 20.0 -> 42.3 t/s 131k change is the overall Turbo deployment comparison against the fresh stock control.

Broader generic gate/up fusion experiments remain opt-in profiling modes. Shared-expert dense gate/up, activation-Q8 cache, gate/up XOR reduction, vec4 weighted-sum, local-weight weighted-sum, down-weighted-sum, atomic down-weighted-sum, SwiGLU activation variants, gate/up rowpack scheduling, gate/up local-Q8 activation caching, gate/up expert-pack scheduling, gate/up dual-dot vecdot, weighted-tail, dispatch guard, gate/up Q8 handoff, exact Q6 down specialization, shared-gate-tail, shared-gate-tail local-gate broadcast, shared-gate-sigmoid-tail, tail-add vec4, tail-add+RMS_NORM, RMS_NORM+MUL, post-norm Q8 handoff, and selective post-norm Q8 handoff probes are also opt-in only after failing to beat the retained default.

Accuracy guardrail for the retained kernel path: Q5_K_M 30-chunk WikiText PPL showed no measured loss in the fresh release-gate run (5.5682 +/- 0.15248 baseline vs 5.5676 +/- 0.15244 candidate), and targeted MUL_MAT_ID backend ops passed 690/690.

Run it (llama.cpp)

bash

# build llama.cpp with the SYCL backend (oneAPI/icpx); see the GitHub repo for the Turbo kernel patch
llama-server -m Nex-N2-mini-B70-Turbo-Q5_K_M.gguf -ngl 99 -fa on -c 131072 \
  -ctk f16 -ctv f16 --jinja --host 127.0.0.1 --port 8090

OpenAI-compatible endpoint at http://127.0.0.1:8090/v1. Nex-N2-mini is a reasoning model — it emits a <think> trace, so give it generous max_tokens. Native recommended sampling (per Nex-AGI): temperature 0.7, top_p 0.95, top_k 40.

Reasoning channel: if the <think> trace shows up inline in the answer instead of a separate reasoning channel, your llama.cpp needs the forced-open <think> parser fix — Nex-N2-mini's template force-opens a bare <think>, and without the fix the server leaves the whole trace in message.content. Apply patches/chat-forced-open-reasoning-leak.patch (git apply) and rebuild.

Provenance & changes (Apache-2.0 §4(b))

Base model: nex-agi/Nex-N2-mini (Apache-2.0), post-trained on Qwen/Qwen3.5-35B-A3B-Base (Apache-2.0).
Changes from the original: converted to GGUF; quantized with a llama.cpp importance matrix (imatrix calibrated on Bartowski calibration_datav3); set GGUF metadata qwen35moe.block_count=40 and qwen35moe.nextn_predict_layers=0 so the model loads in llama.cpp (the MTP/NextN head is speculative-only and absent from the checkpoint — lossless for standard inference).
The weights are quantizations of the original; no other modifications.

License & attribution

⭐ Built on Nex-N2-mini by the Nex-AGI team — their model, their intelligence; this project only quantizes it for Intel Arc. → nex-agi.com · GitHub · HuggingFace

Released under the Apache License 2.0, inherited from the base model. Retain the attribution above. See NOTICE for the full attribution chain (Qwen → Nex-AGI → these quants; plus llama.cpp / MIT).

bibtex

@misc{qwen3.5,
  title  = {{Qwen3.5}: Towards Native Multimodal Agents},
  author = {{Qwen Team}},
  month  = {February},
  year   = {2026},
  url    = {https://qwen.ai/blog?id=qwen3.5}
}

Please also credit Nex-AGI's Nex-N2-mini (https://github.com/nex-agi/Nex-N2) as the base model.

Frosty40/Nex-N2-mini-B70-Turbo-GGUF

NexN2 B70 Turbo — Nex-N2-mini GGUF

Files

Benchmarks (Intel Arc Pro B70, real measurements)

Run it (llama.cpp)

Provenance & changes (Apache-2.0 §4(b))

License & attribution

Frosty40/
Nex-N2-mini-B70-Turbo-GGUF