Native compressed-tensors FP8 checkpoint converted from gorbatjovy/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-heretic at revision 3504178f8097e4a593cdb8c94ce6752037d41f91.

This conversion preserves the abliterated source model, its multimodal vision tower, and its restored Qwen3.5 MTP draft layer. The source model card reports 4 refusals out of 100 prompts and KL divergence 0.0074 for its Heretic 1.4 conversion.

Quantization

Format: compressed-tensors / FP8_BLOCK
Quantized weights: FP8 E4M3, static 128 x 128 blocks
Activations: dynamic FP8, 128-element groups
Preserved in BF16: embeddings, lm_head, vision tower, Gated DeltaNet linear-attention layers, and MTP layers
Checkpoint size observed by vLLM: 34.25 GiB
Converter: llmcompressor==0.9.0, compressed-tensors==0.13.0

Do not force --quantization fp8: vLLM reads quantization_config and automatically selects compressed-tensors.

Validation

Validated locally on an NVIDIA GeForce RTX 5090 (SM120) with vLLM 0.26.0, PyTorch 2.11.0, and CUDA 13.0:

vLLM selected CutlassFp8BlockScaledMMKernel for CompressedTensorsW8A8Fp8.
OpenAI-compatible chat completion: passed.
Automatic function calling with qwen3_coder: passed.
Multimodal OpenAI image input and vision response: passed.
MTP speculative decoding: passed; 7/7 sampled draft tokens accepted.
Refusal-marker smoke test: 0/12 sampled prompts. Generated bodies were discarded and were not stored.
782 explicitly preserved tensors matched the source exactly.
All 15 restored MTP tensors matched the source exactly.
No source tensor was missing from the converted checkpoint.

See fp8-verification.json and SHA256SUMS for machine-readable results and shard checksums.

vLLM

bash

vllm serve ethanfel/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-heretic-FP8 \
  --host 0.0.0.0 \
  --port 8000 \
  --served-model-name qwen3.6-27b-fable-heretic-fp8 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}'

For clients such as ComfyUI OpenAI-compatible nodes, use:

text

Base URL: http://SERVER_IP:8000/v1
Model:    qwen3.6-27b-fable-heretic-fp8

The checkpoint itself supports up to 262,144 tokens. The example deliberately uses 131,072 to leave generous deployment headroom. A client must keep input_tokens + max_tokens <= MAX_MODEL_LEN; do not request the entire remaining context as output without reserving a margin for tokenizer differences.

Behavior and use

This checkpoint inherits the uncensored/abliterated behavior of its source and is not a safety-aligned model. Operators are responsible for access control, logging, applicable law, and any application-level safeguards required by their deployment.