This repository hosts the MTP-oriented llama.cpp/GGUF builds for GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER. The source checkpoint is the full safetensors model in GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER.

These files are rebuilt separately from the first GGUF release using a fresh MTP-aware conversion path. Use these artifacts when your runtime supports Qwen MTP / multi-token prediction acceleration.

Image/video sidecars: This repository now includes the restored Qwen3.6 multimodal config, processor/preprocessor files, tokenizer/chat template, safetensors index, and model-vision-from-qwen3.6-base.safetensors visual tower sidecar. The existing .gguf binaries were not rewritten in this metadata-copy pass.

Current Status

Files are published only after the rebuilt F16 GGUF verifies as MTP/NextN-capable from its actual metadata/tensor layout. Qwen HF tensors named mtp.* are remapped by the MTP-aware llama.cpp converter into GGUF blk.*.nextn.* tensors plus the nextn_predict_layers metadata key. The upload worker refuses to publish quants until the verified F16 marker exists.

Release Snapshot

Item	Value
Source checkpoint	`GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER`
Base model	`Qwen/Qwen3.6-35B-A3B`
Format	GGUF for llama.cpp-compatible runtimes
Conversion target	MTP-aware GGUF export from the llama.cpp MTP branch
Quantization range	F16, Q8_0, Q6_K, Q5_K_M, Q5_K_S, Q4_K_M, Q4_K_S, Q3_K_L, Q3_K_M, Q3_K_S, Q2_K
Final source compliance	98.33% on HarmBench-300
Final source KLD	0.025383937664711
BFCL average plotted improvement	+2.87 percentage points

Benchmark Plots

BFCL Tool-Calling Check

The source safetensors checkpoint was compared against Qwen/Qwen3.6-35B-A3B on a 40-case BFCL subset: 20 simple and 20 multiple-function prompts. GGUF files inherit from that checkpoint, but individual quants should be rechecked if exact tool-call behavior matters.

Metric	Base	NSC-ACE SABER source
Tool-call rate	92.50%	95.00%
Function name accuracy	92.50%	95.00%
Required argument name accuracy	90.00%	93.12%
Required argument value accuracy	79.79%	83.54%
Exact required-call accuracy	75.00%	77.50%

Available Files

File	Status	Notes
`Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-F16.gguf`	uploaded	Full GGUF conversion source / highest local fidelity
`Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q8_0.gguf`	uploaded	Near-full quality, large local file
`Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q6_K.gguf`	uploaded	High-quality local default if memory allows
`Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q5_K_M.gguf`	uploaded	Strong quality/size balance
`Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q5_K_S.gguf`	uploaded	Smaller Q5 option
`Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q4_K_M.gguf`	uploaded	Common balanced local target
`Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q4_K_S.gguf`	uploaded	Smaller Q4 option
`Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q3_K_L.gguf`	uploaded	Lower-memory Q3 option
`Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q3_K_M.gguf`	uploaded	Smaller Q3 balance
`Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q3_K_S.gguf`	uploaded	Small Q3 option
`Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q2_K.gguf`	uploaded	Minimum-size target; quality loss expected

The uploader refreshes this card as each artifact finishes. Uploaded non-F16 files are deleted from the build pod after upload to stay under the pod volume quota.

Which Quant Should I Use?

Quant	Best fit
F16	Maximum fidelity when disk/RAM are not a concern
Q8_0	Very high fidelity local inference
Q6_K	Recommended high-quality local starting point
Q5_K_M	Strong balance for quality and size
Q4_K_M	Practical default for constrained machines
Q3_K_M / Q3_K_S	Low-memory experiments
Q2_K	Smallest target; use only when memory is the hard constraint

For agentic/tool-calling workloads, prefer Q6_K, Q5_K_M, or Q4_K_M when possible. Very low quants can affect formatting, argument fidelity, and refusal calibration.

MTP Notes

These are separate MTP-oriented exports; do not assume the original GGUF repo exposes MTP behavior in runtimes that require MTP metadata/tensors.
MTP speedups depend on runtime support. Use a current llama.cpp build.
Quantized body weights keep blk.*.nextn.* tensors at Q8_0 where supported, because draft-head quality affects speculative acceptance.
The source model's final release metrics are measured before quantization.
Quantized files should be re-evaluated if exact compliance/KLD behavior matters.

Running With llama.cpp

shell

llama-cli \
  -m Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q5_K_M.gguf \
  -c 32768 \
  -ngl 999 \
  -p "Write a compact tool plan for indexing a Python repo."

For OpenAI-compatible local serving: