Views
No views yet
| Metric | Value |
|---|---|
| File size | 14.71 GB (3.56 BPW) |
| Base quant | IQ3_S with custom imatrix |
| Tensor overrides | 317 rules (Q5_K/Q8_0 critical paths) |
| Gen speed (ik_llama sm120, ncmoe=2) | 98.6 tok/s |
| Gen speed (stock, ncmoe=4, 2 slots) | 72.1 tok/s (solo) / 45.2 tok/s (dual) |
| Prompt processing | 228 tok/s (solo), 1357 tok/s (ik_llama) |
| Context | up to 128K (2 slots) or 196K (3 slots) |
| Tool-calling | 20/20 PASS |
| KV cache sweet spot | q8_0 keys + q4_0 values |
| Custom-mix | RAMP-v2 | |
|---|---|---|
| Size | 14.71 GB | 15.2 GB |
| BPW | 3.56 | 3.78 |
| Method | Manual 317 overrides | Automated RAMP pipeline |
| imatrix | From Q4_K_M proxy | From RAMP-v1 bootstrapped |
| Multi-slot | 2-3 slots stable | 1 slot optimized |
| Speed (ik_llama) | 98.6 tok/s | 90 tok/s |
| Tensor category | Quant type | Count | Why |
|---|---|---|---|
| MoE experts (ffn_gate/up/down_exps) | IQ3_S | ~120 | 93% of params, sparse activation |
| Attention + shared experts | Q5_K | ~203 | Critical path, always active |
| SSM alpha/beta/out, attn_v, output | Q8_0 | ~111 | Highest sensitivity |
| Norms, SSM conv1d/dt/a | F32 | ~301 | Small, must be exact |
llama-quantize --imatrix --custom-q and --prune-layers 40 (MTP removed).1# With ik_llama (fastest)
2./llama-server -m Qwen3.5-35B-A3B-IQ3_S-custom-mix.gguf \
3 -ngl 99 --n-cpu-moe 2 -np 1 -c 65536 \
4 --cache-type-k q8_0 --cache-type-v q4_0
5
6# With stock llama.cpp (multi-slot)
7./llama-server -m Qwen3.5-35B-A3B-IQ3_S-custom-mix.gguf \
8 -ngl 99 --n-cpu-moe 4 -np 2 -c 131072 \
9 --cache-type-k q8_0 --cache-type-v q4_0