This is the vLLM version of DavidAU's Qwen3.6-27B Fable-Fusion 711. The weights are FP4. Everything sensitive stays BF16. The repo is 28 GB instead of 56. It serves as an OpenAI-compatible endpoint on any NVIDIA GPU with about 32 GB of VRAM, Ampere or newer. Blackwell is not required.
Scheme
NVFP4A16 weight-only, compressed-tensors format, native vLLM support
Source
quantized directly from the BF16 safetensors. No GGUF step in the lineage.
Kept in BF16
all 48 linear-attention (DeltaNet) layers, the vision tower, the MTP drafter, lm_head
Size
27.7 GB weights (BF16 parent: 55.6 GB)
VRAM
~32 GB minimum; tested on 1x RTX PRO 6000 Blackwell (96 GB)
Context
up to 262,144 tokens, VRAM permitting
Vision
included, unquantized
MTP speculative decoding
verified: 1.56x decode speedup at draft depth 5
FP8 KV cache
calibrated k_scale/v_scale included; verified with --kv-cache-dtype fp8
Tested on vLLM 0.26.0. The server selects the Marlin NVFP4 kernel and warns that the GPU lacks native FP4 compute. The warning is expected: activations are BF16 by design, so the FP4 tensor-core path does not apply.
For faster decoding, enable the built-in MTP drafter:
DavidAU tuned this model for specific sampler settings. They translate to the OpenAI API as follows:
python
1from openai import OpenAI
2client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")34# Thinking mode (default template): creative work, reasoning5r = client.chat.completions.create(6 model="kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16",7 messages=[{"role":"user","content":"Open a noir story set in a lighthouse."}],8 temperature=1.0, top_p=0.95, max_tokens=2048,9 extra_body={"top_k":20,"min_p":0.0,"repetition_penalty":1.0},10)1112# Coding: temperature=0.6, same top_p and top_k13# Instruct mode: temperature=0.7, top_p=0.80, presence_penalty=1.5
The repo ships two chat templates. chat_template.jinja (thinking) is the default. For instruct-style responses without a thinking block, pass --chat-template pointing at chat_template-instruct.jinja at serve time.
This repo or the GGUF?
llama.cpp, LM Studio, Ollama, one user on a consumer card: use DavidAU's GGUF repo.
vLLM, concurrent requests, an OpenAI-compatible server, or 100k+ context: use this repo.
At the time of publication this is the only quant of Fable-Fusion 711 that stock vLLM can serve. The other safetensors quants of this model use a custom runtime format.
Fidelity
Measured with lm-evaluation-harness 0.4.12 (vLLM backend, 0-shot, batch auto) against the BF16 parent on the same hardware:
Task
BF16
NVFP4A16
Delta
ARC-Challenge (acc_norm)
0.6126
0.6203
+0.008
HellaSwag (acc_norm)
0.8502
0.8490
-0.001
Winogrande (acc)
0.7806
0.7814
+0.001
Deltas sit inside the standard error on every task (ARC-Challenge stderr is 0.014). The quantization is a data-free RTN pass. No calibration dataset was used, so nothing was tuned toward any benchmark. Absolute scores depend on harness settings; the point of this table is the like-for-like comparison, both configs measured 0-shot on the same harness and hardware.
Speed
Single request, 512-token greedy generations, 1x RTX PRO 6000 Blackwell (96 GB), vLLM 0.26.0:
Config
Decode speed
Standard decoding
56.2 tok/s
MTP speculative, depth 5
87.4 tok/s (1.56x)
Acceptance across the test: 996 of 3000 drafted tokens (about 1.7 extra tokens per draft window). Throughput under concurrent load will differ.
Only the Linear weights in the full-attention and MLP blocks carry FP4. The DeltaNet linear-attention layers, the vision tower, the MTP drafter, and lm_head keep BF16, because hybrid-attention layers and drafters degrade badly under 4-bit quantization. The exact recipe ships in recipe.yaml.
Packaging notes
The parent repo ships no image-processor configs. preprocessor_config.json and video_preprocessor_config.json here come from Qwen/Qwen3.6-27B.
The 15 MTP drafter tensors live in model-mtp-bf16.safetensors and are wired into model.safetensors.index.json. Transformers drops them on load; they were re-extracted from the parent checkpoint so speculative decoding remains possible.
Content notice
The parent model is refusal-ablated ("Uncensored", "Heretic") and tuned for unconstrained creative writing. This quant changes none of that. It will produce content that aligned models refuse. You are responsible for how you deploy it.
Caveats
The vision path loads but has not been benchmarked.
Lineage and attribution
Qwen/Qwen3.6-27B by the Qwen team, then the Fable-Fusion 711 multi-stage tune and merge by DavidAU with collaborators credited on the parent card, then this quantization. Thanks to all of them. Apache-2.0 throughout.
