A real, self-calibrated 4-bit GPTQ quantization of deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B, produced and measured by QuantThink — a reproducible benchmark measuring how quantization affects small reasoning (long chain-of-thought) models on a 4GB consumer GPU (RTX 3050 Laptop).

Reproduce:

Real evaluation against this project's frozen GSM8K subset (N=6 problems x 2 seeds = 12 samples — same disclosed small-N first-pass discipline as every other result in this project; see docs/RUN_REAL.md):

Loaded and evaluated via gptqmodel's own BACKEND.TORCH inference path (pure PyTorch — the Marlin/Triton kernels need nvcc to JIT-compile, which this driver-only machine doesn't have; see the reproduce command below). Note the VRAM figure here is measured via torch.cuda.max_memory_allocated(), not the nvidia-smi-based measurement used for the GGUF quants elsewhere in this project — the two aren't directly comparable without accounting for that methodology difference.

For context, this project's llama.cpp Q4_K_M GGUF quant of the same base model measured 0.583 Acc / 420.8 TL / 1532.1 CTS at the same sample size (see happynood/DeepSeek-R1-Distill-Qwen-1.5B-GGUF) — this GPTQ quant nominally scores higher on this small sample, plausibly because GPTQ uses real calibration data to minimize reconstruction error while GGUF's Q4_K_M is a calibration-free (round-to-nearest-style) scheme. This is a single N=12 comparison, not a statistically confirmed result — no bootstrap CI has been computed at this sample size.