You can deploy this model and start earning money today!

Zynerji/
Qwen3.8-27B-PristinelyUncensored-HOMEUSER-16-24

Name: Qwen3.8-27B-PristinelyUncensored-HOMEUSER-16-24
Rating: 2.5 (813 reviews)

Views

No views yet

Qwen3.8-27B-PristinelyUncensored-HOMEUSER-16-24

A 27B hybrid-attention VLM with vision and speculative decoding intact, in 14.75 GB. Every number below was measured on that GPU, on rented hardware, with the flags shown further down. Nothing is extrapolated.

Every GPU measured

GPU	VRAM	max context	tok/s @4k	MTP k=2	TTFT 3.5k
RTX 5090	32 GB	262,144	87.6	0.92x	0.98 s
RTX 4090	24 GB	262,144	56.4	0.98x	1.30 s
RTX 3090 Ti	24 GB	262,144	56.1	1.49x	2.40 s
RTX 3090	24 GB	262,144	50.4	1.11x	—
RTX A4000	16 GB	14,336	28.3	✗ fails	3.78 s
RTX 5060 Ti	16 GB	12,288	27.4	✗ fails	4.49 s
RTX 5070 Ti	16 GB	10,240	50.8	✗ fails	2.25 s
RTX 4080 Super	16 GB	not cleanly measured	43.6	✗ fails	2.00 s

✗ fails = the engine will not start with MTP enabled — no headroom for the draft model on 16 GB. — = not measured. TTFT is time-to-first-token on a real 3,500-token prompt, not an empty one.

262,144 is the model's max_position_embeddings — on 24 GB and above there is no context left to give, confirmed on four independent machines across three GPU generations.

The 16 GB ceiling is card-specific. Three cards whose visible VRAM differs by 0.16 GiB landed 4,096 tokens apart, so measure yours rather than trusting a single number — anything held by a display server or compositor comes straight off it. 16,384 fit on no 16 GB card tested.

⚠️ READ FIRST — install the fork

This model runs on Zynerji/vllm, not on stock vLLM:

pip install git+https://github.com/Zynerji/vllm@ektome-homeuser-v2

Pinned commit: 29f0f1c61f4f4ddcb553de92e6893c9524a93b80

Why it is required. This is the only Qwen3.8-27B quant that compresses embed_tokens and lm_head — two 248320 x 5120 tensors that every other release leaves in BF16. That is where the ~6 GB of headroom comes from, and it is why 262,144 tokens of context fit on a 24 GB card at all.

Stock vLLM cannot load them. qwen3_5.py constructs VocabParallelEmbedding without passing quant_config, so a quantized embedding is built unquantized and load dies with no module or parameter named embed_tokens.weight_packed. The identical omission appears a second time in qwen3_5_mtp.py, so the MTP speculative head cannot start either (... in Qwen3_5MultiTokenPredictor). llama.py already passes both arguments; the Qwen paths do not. The fork fixes all three sites.

The change is small and unencumbered — anyone is welcome to lift it:

python

# vllm/model_executor/models/qwen3_5.py  AND  qwen3_5_mtp.py
self.embed_tokens = VocabParallelEmbedding(
    self.vocab_size,
    config.hidden_size,
    quant_config=quant_config,                         # <-- add
    prefix=maybe_prefix(prefix, "embed_tokens"),       # <-- add
)

Run it

Two flags are load-bearing on every card. --max-num-batched-tokens 2048 is pure activation memory (the 8192 default costs you 192k->256k on 24 GB and drops 16 GB to 4k), and --gpu-memory-utilization must not exceed 0.97 — at 0.98 the engine dies with EngineDeadError at contexts that succeed at 0.97, so utilization is non-monotonic here.

24 GB card — the long-context configuration:

bash

vllm serve Zynerji/Qwen3.8-27B-PristinelyUncensored-HOMEUSER-16-24 \
  --max-model-len 262144 \
  --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.97 \
  --max-num-batched-tokens 2048 \
  --max-num-seqs 16

Every flag here is load-bearing. Drop --max-num-batched-tokens 2048 and the ceiling falls to 192k; raise utilization to 0.98 and the engine dies with EngineDeadError.

16 GB card — it fits, but only just:

bash

vllm serve Zynerji/Qwen3.8-27B-PristinelyUncensored-HOMEUSER-16-24 \
  --max-model-len 10240 \
  --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.97 \
  --max-num-batched-tokens 2048 \
  --max-num-seqs 1

--kv-cache-dtype fp8 and --max-num-batched-tokens 2048 are both mandatory on 16 GB — at the 8192 default the ceiling collapses to 4k.

The 16 GB ceiling is card-specific. Measure yours; do not assume 12,288. Cards whose visible VRAM differs by 0.16 GiB landed 4,096 tokens apart:

card	VRAM visible to vLLM	max context	tok/s @4k	TTFT 3.5k
RTX A4000	15.63 GiB	14,336	28.3	3.78 s
RTX 5060 Ti	15.48 GiB	12,288	27.4	4.49 s
RTX 5070 Ti	15.47 GiB	10,240	50.8	2.25 s

Walk --max-model-len down from 16384 in 2048 steps until the engine starts; anything left over by a display server or compositor comes straight off this number. 16,384 did not fit on any 16 GB card tested. If you need long context, use a 24 GB card — the 16 GB tier is "a 27B that fits", not a long-context machine.

Do not enable MTP on a 16 GB card. There is no headroom for the draft model at util 0.97 and the engine fails to start (verified on 5060 Ti, 5070 Ti, 4080 Super and A4000 — every 16 GB card tested, across two independent sweeps).

--max-num-seqs matters on this architecture. 48 of the 64 layers are linear-attention (gated-delta) and each concurrent decode sequence consumes one Mamba state block. On 24 GB there are ~39 blocks, and vLLM refuses to start above that:

ValueError: max_num_seqs (128) exceeds available Mamba cache blocks (39).

Concurrency is bounded by Mamba state, not by KV cache. Keep max_num_seqs <= 32 on 24 GB.

Measured results

RTX 3090 (24 GB), vLLM 0.27.1, this exact checkpoint.

Benchmark	Result	n
MMLU	0.7995	1531
Compliance (refusal rate)	0.988 — 5 refusals	416
HumanEval pass@1 (by execution)	0.8902	164
MMStar (vision)	0.6633	300

HumanEval and MMStar were measured on the identical checkpoint with a BF16 lm_head; the int4 lm_head shipped here scores MMLU 0.7995 vs that build's 0.7975 at the same n=1531 — a tie, since an unpaired test at this n needs ~2.8 points to resolve.

Throughput and memory

card	context	KV	VRAM	tok/s
24 GB	262,144	fp8	22.97 / 23.56 GiB	7.4 eager · 50.6 graphs
24 GB	2,048	fp8	20.0 GiB	50.3 (b1) / 341.9 (b8)
16 GB	10,240–14,336	fp8	~15.3 / 15.5 GiB	27–51

Use CUDA graphs. With enforce_eager the same 256k configuration runs at 8.5 tok/s; with graphs enabled (the default) it runs at 50.6 tok/s — a 6x difference, and the single biggest performance factor after the flags above.

Long context costs prefill, not decode. Measured time-to-first-token with the cache genuinely full: ~26 s at 30k, ~2.6 min at 125k, ~7.3 min at 250k (prefill throughput falls from ~1,150 to ~575 tok/s as the quadratic attention term grows). If you query a stable corpus repeatedly, enable prefix caching so that cost is paid once rather than per request.

Only the 16 full-attention layers grow with context — 64 KB/token bf16, 32 KB/token fp8. The other 48 layers hold constant-size state. That is why 256k costs ~8 GiB of KV instead of the ~100 GiB a dense 27B would need.

What "mixed precision" means here

985 modules, 14.75 GB. Every group sits at the precision its runtime kernel can serve — the constraints are geometric, not stylistic.

component	precision	why
decoder (658 Linears)	int4 g128, GPTQ-calibrated	`MarlinLinearKernel`; all 400 2-D decoder weights satisfy Marlin tiling
vision tower (110 Linears)	int4 g16, RTN	`linear_fc2` has in_features 4304 = 16x269; 16 is the only group size under 128 that divides it, and only `HummingLinearKernel` accepts 16
`embed_tokens`	int4 g128 symmetric	vLLM's embedding path requires INT + symmetric
`lm_head`	int4 g128 symmetric	needs symmetric and a per-group `format: pack-quantized`; asymmetric int4 finds no scheme
mamba `in_proj_a/b`, `conv1d`	BF16, 137.6 MB	out_features=48; 48 % 64 != 0 fails Marlin tiling
MTP head (15 tensors)	FP8 + BF16 norms	round-trip verified, rel-L1 0.0224–0.0226
quantization scales	BF16, 399.7 MB	irreducible metadata, 2.7% of the file

lm_head quantisation error is rel-L1 0.1184, lower than the embedding that ships alongside it (0.173).

Multi-token prediction

The 15-tensor MTP head is retained and FP8-quantised rather than dropped:

--speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'

Speculative decoding is exact at temperature 0 — drafts are verified against the target model, so this changes throughput and never quality.

MTP is card-dependent — benchmark it on your own GPU before leaving it on. Measured at 4k, batch 1, CUDA graphs on, across a 10-GPU sweep:

GPU	no spec	MTP k=2	gain
RTX 3090 Ti	56.1	83.4	1.49x
RTX 3090	50.4	55.9	1.11x
RTX 4090	56.4	55.1	0.98x
RTX 5090	87.6	80.4	0.92x
any 16 GB card	—	—	fails to initialise

Ampere gains; Ada and Blackwell lose. On 16 GB there is no headroom for the draft model at util 0.97 and the engine will not start with MTP enabled — leave it off there.

Gains also depend on how predictable the output is (structured text and code accept more drafts than free prose) and shrink as concurrency rises. MTP requires the fork's qwen3_5_mtp patch; without it the engine fails to start on any card.

Vision

The tower is retained and functional (MMStar 0.6633, vs 0.6500 for an FP8 tower on the same items — int4-g16 costs nothing measurable). Pass enable_thinking=False for classification-style prompts: this checkpoint's template injects a high reasoning-effort preamble, and without the flag the model spends its token budget reasoning instead of answering.

Provenance and limits

From Zynerji/Ektome-Qwen3.8-27B-PristinelyUncensored (Ektomē refusal-direction excision), itself from Qwen/Qwen3.8-27B. Structure verified complete against the parent: 985/985 modules.

Uncensored. ~99% compliance means it will answer almost anything asked of it. You own what you generate with it.
MMLU 0.7995 is not separable from the parent line's 0.7975 — treat them as equivalent, not better.
The vision comparison is n=300 and within noise: it supports "no degradation", not "better".
256k fits ONLY with --max-num-batched-tokens 2048; at the 8192 default the ceiling is 192k.
On 16 GB the ceiling is 12k (16k does not fit). Long context needs a 24 GB card.
Requires the fork above until upstream lands the embed_tokens fix.