Views
No views yet
| File | Size | Notes |
|---|---|---|
Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved-TQ3_4S.gguf | ~14 GB | Mixed K-quant + TQ3_4S layout |
A multimodalmmprojis not included in this repo. If you need vision, grab the matchingmmproj-BF16.gguffrom the base model repo below.
llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved-GGUF — the BF16 GGUF + mmproj were the quantization input.YTan2000/Qwen3.6-35B-A3B-MTP-TQ3_4S — the mixed-precision tensor layout below was matched to this model.turbo-tan/llama.cpp-tq3 — llama.cpp fork that adds the TQ3_4S type and --spec-type draft-mtp speculative decoding.TQ3_4S would quantize all weights at 4 bpw (~18 GB). Instead this is a
mixed layout (base ftype Q3_K_M, with per-tensor overrides) matching the
reference model:token_embd = Q5_K
output = Q4_K
attn_* / *_shexp / ssm_out = Q6_K
ffn_gate_exps / ffn_up_exps = Q2_K
ffn_down_exps = Q3_K (Q4_K on blocks 21, 28, 38)
ssm_alpha / ssm_beta / nextn.eh_proj = TQ3_4Sllama-quantize:1llama-quantize \
2 --token-embedding-type Q5_K \
3 --output-tensor-type Q4_K \
4 --tensor-type-file tensor_types.txt \
5 Qwen3.6-35B-A3B-...-BF16.gguf \
6 Qwen3.6-35B-A3B-...-TQ3_4S.gguf \
7 Q3_K_Mllama-serverturbo-tan/llama.cpp-tq3
fork (for TQ3_4S + MTP speculative decoding). Tuned for a 16 GB GPU; adjust
-ngl, --ctx-size, and --cache-ram to your hardware.1llama-server \
2 --host 0.0.0.0 --port 8080 \
3 --model Qwen3.6-35B-A3B-...-TQ3_4S.gguf \
4 --jinja \
5 --chat-template-file chat_template.jinja \
6 -ngl 55 \
7 -fa on \
8 -ctk q8_0 -ctv tq3_0 \
9 --batch-size 2048 \
10 --ubatch-size 512 \
11 --ctx-size 64000 \
12 --parallel 1 -np 1 \
13 --spec-type draft-mtp \
14 --spec-draft-ngl 99 \
15 --spec-draft-n-max 2 \
16 --spec-draft-n-min 1 \
17 --spec-draft-p-min 1.0 \
18 --spec-draft-type-k q4_0 \
19 --spec-draft-type-v tq3_0 \
20 --reasoning on \
21 --reasoning-format auto \
22 --warmup --perf \
23 --threads 4 --threads-batch 8 \
24 --cache-ram 16000 \
25 --ctx-checkpoints 32--ctx-size 64000 is roughly the empirical max on 16 GB before OOM; lower it if you hit memory limits.-ctk q8_0 -ctv tq3_0 quantizes the KV cache to fit more context.--spec-type draft-mtp uses the model's native MTP head as the speculative draft — no separate draft model needed.--mmproj mmproj-BF16.gguf (runs on CPU).--reasoning on with --reasoning off --reasoning-budget 0.