Qwen3-32B-AWQ-INT4 – AI Model by drawais | AlphaNeural AI

You can deploy this model and start earning money today!

drawais/
Qwen3-32B-AWQ-INT4

like

0

text-generation

Views

No views yet

Qwen3-32B-AWQ-INT4

INT4 quantization of Qwen/Qwen3-32B. Built to run on a single 24 GB+ GPU.

Footprint


Source params	32B
Quantized weights	~18 GB on disk
Inference VRAM (incl. KV cache @ 32K context)	~24 GB

Fits any 24 GB+ GPU: RTX 3090 / 4090 / 5090, A5000, A6000, A100 40GB, etc.

Bench

Scored on drawais/needle-1M-bench-mvp (50K-token haystack, real arxiv text):

Metric	Score
Overall recall	100.0%
Paper-anchored	100.0%
Synthetic codes	100.0%

Quick start

vllm serve drawais/Qwen3-32B-AWQ-INT4 --quantization awq_marlin --max-model-len 32768

python

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("drawais/Qwen3-32B-AWQ-INT4")
model = AutoModelForCausalLM.from_pretrained("drawais/Qwen3-32B-AWQ-INT4", device_map="auto")

Context length

Native: 40,960 tokens (inherits from base model). For longer contexts, enable YaRN rope-scaling per the base model's config.

License

Apache 2.0 (inherits from base model).