FP8 (block 128×128, E4M3) quantization of an abliterated Qwen3.8-27B, with the vision tower and the working MTP speculative-decoding head both preserved.

What "abliterated" means here. The base model has had its refusal direction removed — it will not decline requests the way the official Qwen3.8-27B does. That modification comes from the base model, not from this repository; this repo contributes the FP8 quantization only. The base is Apache-2.0 and publicly available. Use accordingly.

Why this build

Surveying 148 abliterated/uncensored Qwen3.8-27B repositories in August 2026 turned up exactly one FP8 build, and it was access-gated. Everything else was GGUF, NVFP4 (Blackwell-only), AWQ, or unquantized BF16. On Hopper hardware — where FP8 has native tensor-core support — that left no ungated option.

Only a handful of builds in any format preserve the MTP head, and getting it to actually work under vLLM required the checkpoint to be in the native fp8 format rather than compressed-tensors (see below).

Measured against the official build

Same hardware (H100 47GB vGPU), same vLLM 0.27.1, same flags.

	`Qwen/Qwen3.8-27B-FP8`	this build
Thai extraction, exact codepoint (temp 0, 4 runs)	16/16	16/16
Coding tasks — generated code executed against assertions	4/5	4/5
Vision: Thai table, activity names read exactly	4/4	4/4
Vision: numeric cells	correct	correct
MTP acceptance rate	58%	52%
tok/s @ concurrency 1	131	132
tok/s @ concurrency 8	816	813
tok/s @ concurrency 16	—	1,373

The one coding task both models fail is the same one, for the same reason: the model spends its whole token budget reasoning and never emits an answer.

Hardware requirements

FP8 needs tensor-core support to be worth anything. On architectures without it, vLLM either refuses the checkpoint or falls back to a dequantize-then-multiply path that is slower than a 4-bit AWQ build.

Architecture	Examples	Compute cap.	FP8 tensor cores	Runs this model
Volta	V100	7.0	✗	no — also lacks BF16 tensor cores, and vLLM has dropped Volta
Turing	RTX 20xx, T4	7.5	✗	no
Ampere	RTX 30xx, A100, A40, A6000	8.0 / 8.6	✗	see note below
Ada Lovelace	RTX 4090, L4, L40S	8.9	✓	yes, if VRAM allows
Hopper	H100, H200, H20	9.0	✓	yes — tested here
Blackwell	RTX 50xx, B100/B200, GB200	10.0+	✓ (plus FP4)	yes

On Ampere (A40, A100, A6000). These have the VRAM — an A40 has 48 GB, the same as the H100 47C this was tested on — but no FP8 tensor cores. vLLM can fall back to Fp8MarlinLinearMethod, which keeps weights in FP8 and dequantizes inside the kernel, so memory still works out. Whether that path accepts block-wise 128×128 scales (as opposed to per-tensor or per-channel) is untested here — we had no Ampere card to try it on. If you are on Ampere, an INT8 W8A16 or AWQ build of the same base is the safer choice.

VRAM

Weights occupy ~30 GB. The rest goes to KV cache, which decides how much context you can actually serve.

VRAM	Verdict	KV pool at `--kv-cache-dtype fp8`
24 GB (RTX 4090)	will not load on one card	—
32 GB (V100 32GB)	no — wrong architecture entirely	—
48 GB (A40)	VRAM is fine; FP8 path is not — see note above	—
47–48 GB (H100 47C vGPU, L40S)	works	~350k tokens — measured on H100 47C
80 GB (H100/H200 80GB)	comfortable	~600k+ tokens
2× 24 GB with `--tensor-parallel-size 2`	works on Ada	splits weights ~15 GB per card

At 47 GB the whole 262,144-token context fits for a single request with room for roughly one more — Maximum concurrency for 262,144 tokens per request: 1.3x. Shorter contexts scale proportionally: ~20 concurrent requests at 16k each.

Without --kv-cache-dtype fp8, KV for 262k context needs 16.17 GiB instead of ~8.5 GiB and a 47 GB card will refuse to start.

Serving

bash

vllm serve pramoths/Qwen3.8-27B-Uncensored-FP8-MTP \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.94 \
  --kv-cache-dtype fp8 \
  --max-num-seqs 16 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --enable-prefix-caching \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

--kv-cache-dtype fp8 is required for 262k context on a 47GB card — fp16 KV needs 16.17 GiB and will refuse to start. --max-num-seqs must stay below the available Mamba state block count; the default of 256 fails to boot.

Verify MTP is actually working — it can fail silently:

curl -s localhost:8000/metrics | grep spec_decode_num_accepted_tokens_total

A Mean acceptance length of 1.00 means every draft is being rejected. You are paying the drafting cost for nothing.

What was and was not quantized

Follows Qwen/Qwen3.8-27B-FP8 exactly — the distinction is matmul versus control parameter, not which component of the model:

FP8: q/k/v/o_proj and gate/up/down_proj, in the language layers and in the MTP head.
BF16: LayerNorms, gates, A_log, dt_bias, conv1d, and the gated-DeltaNet in_proj_a / in_proj_b — these govern recurrent state, where quantization error accumulates rather than cancelling.
BF16: the entire vision tower, lm_head, embed_tokens, mtp.fc.

Reproducing

Scripts, benchmarks, and a writeup of the five silent failure modes encountered along the way: https://github.com/pramoth/qwen38-fp8-forge

Quantization needs no calibration data and no GPU — roughly two minutes of CPU.

Attribution

Qwen/Qwen3.8-27B — original model, Apache-2.0
JonathanColetti/Qwen3.8-27B-Uncensored — abliterated base
Qwen/Qwen3.8-27B-FP8 — the quantization recipe this build copies