Nex-N2-mini's original chat template uses complex vision processing macros that stock llama.cpp's Jinja parser cannot handle correctly. This causes [object Object] thinking tags to not be injected, breaking --reasoning-format.

The official fix: Use Nex's patched llama.cpp, which works with the unmodified GGUF and unmodified template. Once Nex's upstream patch is merged into stock llama.cpp, this workaround will no longer be needed.

⚠️ Do NOT modify chat_template.jinja. The model was trained strictly on the current template — editing the tags deviates from the training-time format and may degrade output quality. See discussion #3.

💡 What is APEX?

These GGUF files are quantized using APEX, a MoE-aware mixed-precision quantization technique that outperforms standard quantization methods while being significantly smaller.

APEX beats Q8_0 perplexity at half the size — and even beats F16.

APEX classifies every tensor by its role — routed expert, shared expert, or attention — and applies a layer-wise precision gradient, giving the most sensitive edge layers higher precision and compressing the redundant middle layers more aggressively.

📦 Available Files

File	Size	BPW	Note
`Nex-N2-mini.BF16.gguf`	64.6 GB	16.0	Full precision reference
`Nex-N2-mini-APEX-Quality.gguf`	21.3 GB	5.23	Highest quality, best accuracy
`Nex-N2-mini-APEX-Balanced.gguf`	23.6 GB	5.85	Best all-rounder, recommended
`Nex-N2-mini-APEX-Compact.gguf`	15.4 GB	3.81	Best quality/size ratio, 16GB VRAM
`mmproj-Nex-N2-mini.F16.gguf`	858 MB	-	Vision projector (required for image/video)

🧠 Model Details

Architecture	Qwen3.5 MoE (GatedDeltaNet + Full Attention) + Vision Encoder
Parameters	35B total, 3B active per token
Experts	256 routed experts, 8 active per token
Layers	40 layers (30 linear_attn + 10 full_attn)
Context	262,144 tokens
Vision	Image support (mmproj 858MB)
Thinking	Qwen3-style think tags — requires Nex's patched llama.cpp (see above)

🚀 Usage

Download Nex's patched llama.cpp

Binaries: nex-agi/llama.cpp | Docker: ghcr.io/nex-agi/llama.cpp:server-cuda-nex-b9596-fix-b9598-8c0d5c9

./llama-server \ -m Nex-N2-mini-APEX-Quality.gguf \ -ngl 99 -ncmoe 19 -c 32768 \ --host 0.0.0.0 --port 8081

Replace Nex-N2-mini-APEX-Quality.gguf with your preferred quantization tier. Add --mmproj mmproj-Nex-N2-mini.F16.gguf for vision. Recommended sampling: temperature 0.7, top_p 0.95, top_k 40, min_p 0.

📋 Original Model Benchmarks

Benchmark	Score	Category
BrowseComp	74.1	Agent
SWE-Bench Verified	74.4	Coding
Terminal-Bench 2.1	60.7	Coding
GPQA Diamond	82.6	Reasoning
IFEval	89.1	Instruction

From the original Nex-N2-mini model card (BF16, full precision).

Using with Stock llama.cpp

If you cannot use Nex's patched llama.cpp, a template-fixed version is available at SC117/Nex-N2-mini-template-fix-APEX-GGUF. These GGUFs have a modified chat_template.jinja embedded so that --reasoning-format works on stock llama.cpp without --chat-template-file.

⚠️ The Nex team explicitly advises against modifying the chat template — the model was trained strictly on the original template, and deviating from the training-time format may degrade output quality. See discussion #3. Use the template-fixed version only if you have no alternative, and be aware of the potential quality trade-off.

SC117/Nex-N2-mini-APEX-GGUF

Nex-N2-mini

Using with Stock llama.cpp

Links

SC117/
Nex-N2-mini-APEX-GGUF