Views
No views yet
Qwen/Qwen3-8B. Built to run on a single 12 GB+ consumer GPU.| Source params | 8B |
| Quantized weights | ~5.7 GB on disk |
| Inference VRAM (incl. KV cache @ 32K context) | ~10 GB |
drawais/needle-1M-bench-mvp (50K-token haystack, real arxiv text):| Metric | Score |
|---|---|
| Overall recall | 80.0% |
| Paper-anchored | 80.0% |
| Synthetic codes | 80.0% |
vllm serve drawais/Qwen3-8B-AWQ-INT4 --quantization awq_marlin --max-model-len 327681from transformers import AutoTokenizer, AutoModelForCausalLM
2tok = AutoTokenizer.from_pretrained("drawais/Qwen3-8B-AWQ-INT4")
3model = AutoModelForCausalLM.from_pretrained("drawais/Qwen3-8B-AWQ-INT4", device_map="auto")