You can deploy this model and start earning money today!

CuTIsolation/
Qwen3.8-27B-W4A8

Name: Qwen3.8-27B-W4A8
Availability: InStock

Views

No views yet

Qwen3.8-27B W4A8 (for ComfyUI)

W4A8 quantized version of Qwen3.5/Qwen3.8-27B (qwen3_5 architecture) as a single safetensors file, loadable directly by ComfyUI. Weights are compressed from 27.9GB (BF16) to 16.7GB.

Model Overview

Base model: Qwen3.8-27B (Apache-2.0)
Quantization format: asym_w4a8_int8 (4-bit weights, int8 activations)
Grouping: weight group_size = 16, convrot_groupsize = 256
Quantization algorithm: comfy-kitchen AsymW4A8Int8Layout
- ConvRot Hadamard rotation (256-blocks) followed by per-group quantization
- Per-group 16-level Lloyd-Max codebook + fp8 group scales + fp32 channel scales
File size: qwen3.8-27b-w4a8.safetensors ≈ 16.7GB

Conversion Scope

Action	Layers
Quantized (int4)	LM linear projections: in_proj_qkv/z/b/a, out_proj, q/k/v/o_proj, gate/up/down_proj, lm_head
Kept BF16	Vision tower, embeddings, norms, DeltaNet conv1d
Dropped	MTP head (not supported by ComfyUI's Qwen35 implementation)

ComfyUI Usage

Put qwen3.8-27b-w4a8.safetensors into ComfyUI/models/text_encoders/.
Select the file in a CLIP Loader. ComfyUI auto-detects it as QWEN35_27B (detect_te_model) and enables mixed-precision layers (llama_detect → llama_quantization_metadata).
Quantized layers run through the AsymW4A8Int8Layout CUDA/Triton kernels.

Accuracy

Dequantizing a single layer (in_proj_qkv) reproduces comfy-kitchen's quantization of the original BF16 weight bit-for-bit (relative error 0.00000, checks passed).
Typical W4A8 forward error vs full-precision BF16 is ~1% (varies by layer and data distribution).
It is recommended to run your own quality evaluation before deploying.

Hardware Requirements

NVIDIA GPU (Ampere or newer, CUDA 13.0+) recommended for the optimized comfy-kitchen CUDA backend; CPU / Triton fall back gracefully.
Conversion measured on RTX 4060 (8GB VRAM) from 52GB BF16 shards: ~4 minutes.

Conversion Tool

Quantized with convert_qwen35_w4a8.py (comfy-kitchen quantization backend), shipped alongside the ComfyUI repo:

bash

python convert_qwen35_w4a8.py \
    --input-dir ./qwen38_27b \
    --output ./qwen3.8-27b-w4a8.safetensors \
    --verify

License & Disclaimer

The base model is Apache-2.0; this quantized model is released under the same license. This is a community quantization and differs from the original weights in accuracy. Use at your own discretion; please comply with the base model's terms of use and applicable laws in your jurisdiction.

CuTIsolation/Qwen3.8-27B-W4A8