DLM-2.0-14B-GPTQ

Name: DLM-2.0-14B-GPTQ
Availability: InStock

Overview

This is a GPTQ 4-bit quantized version of dnotitia/DNA-2.0-14B, optimized for efficient inference by DLM (Data Science Lab., Ltd.).

GPTQ W4A16 quantization reduces model size by approximately 73% while maintaining near-original accuracy. Fully compatible with vLLM for high-throughput production serving, and runnable on consumer-grade GPUs (16GB+).

Model Details

Attribute	Value
Base Model	dnotitia/DNA-2.0-14B
Architecture	Qwen3ForCausalLM
Parameters	~14B
Quantization	GPTQ W4A16
Group Size	128
Quantization Tool	llm-compressor
Calibration Data	HuggingFaceH4/ultrachat_200k (512 samples)
Model Size	~8 GB (vs ~30 GB in BF16)
Context Length	32K native / up to 131K with YaRN
Vocabulary	151,936 tokens
License	Apache 2.0
Quantized By	DLM (Data Science Lab., Ltd.)

Quantization Details

Method: GPTQ (Generative Pre-trained Transformer Quantization)
Precision: W4A16 — 4-bit weights, 16-bit activations
Group Size: 128 (per-group scale for balance between accuracy and compression)
Calibration: 512 samples from HuggingFaceH4/ultrachat_200k (train_sft split), max sequence length 2048
Format: compressed-tensors (safetensors)
Preserved layers: lm_head kept in full precision (BF16)
Targets: All Linear layers (except lm_head)

Quantized Models Comparison

Model	Method	Size	Quality Loss	Min GPU VRAM	Use Case
dataslab/DNA-2.0-14B-FP8	FP8 W8A8	~19 GB	Minimal	24 GB	High-performance server
dataslab/DLM-2.0-14B-GPTQ	GPTQ W4A16	~8 GB	Low	16 GB	Lightweight / consumer GPU

Usage

vLLM (Recommended)

bash

vllm serve dataslab/DLM-2.0-14B-GPTQ \
  --dtype auto \
  --max-model-len 32768 \
  --enable-reasoning \
  --reasoning-parser deepseek_r1

Extended context (up to 131K with YaRN):

bash

vllm serve dataslab/DLM-2.0-14B-GPTQ \
  --dtype auto \
  --rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' \
  --max-model-len 131072 \
  --enable-reasoning \
  --reasoning-parser deepseek_r1

Python (vLLM)

python

from vllm import LLM, SamplingParams

llm = LLM(model="dataslab/DLM-2.0-14B-GPTQ")
sampling_params = SamplingParams(
    temperature=0.6, top_p=0.95, top_k=20, max_tokens=4096
)

messages = [
    {"role": "user", "content": "한국의 경제 발전 과정에 대해 설명해주세요."}
]
outputs = llm.chat(messages, sampling_params=sampling_params)
print(outputs[0].outputs[0].text)

Python (Transformers)

python

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("dataslab/DLM-2.0-14B-GPTQ")
model = AutoModelForCausalLM.from_pretrained(
    "dataslab/DLM-2.0-14B-GPTQ",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "복잡한 윤리적 딜레마에 대해 다각도로 분석해줘."}
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_dict=True, return_tensors="pt"
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=4096,
    temperature=0.6,
    top_p=0.95,
    top_k=20,
    do_sample=True,
)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Dynamic Thinking Mode

This model inherits DNA 2.0's dynamic thinking capability:

Thinking mode: Add /think to enable detailed step-by-step reasoning (temperature=0.6)
Non-thinking mode: Add /no_think for concise, direct responses (temperature=0.7)

Base Model

DNA 2.0 is developed by Dnotitia Inc. and features:

Smoothie Qwen3 foundation with balanced multilingual optimization
Uncensored reasoning training for objective, unbiased responses
Advanced RL post-training for enhanced mathematical reasoning and Korean language capabilities

For more details, see the arXiv paper (2507.05686).

License

Apache 2.0 — Same as the base model.

Quantized and released by DLM (Data Science Lab., Ltd.) — HuggingFace