You can deploy this model and start earning money today!

namgyu-youn/
EXAONE-4.0-1.2B-GPTQ-W4A16-EoRA

Name: EXAONE-4.0-1.2B-GPTQ-W4A16-EoRA
Availability: InStock

Views

No views yet

EXAONE-4.0-1.2B GPTQ W4 + EoRA

4-bit: mse=2.4 + group_size=32 + SmoothMSE(64,0.75) + EoRA(rank=64)

Expected: 98-99.5% quality × 2.8-3.2x = 2.74-3.18 score

python

from gptqmodel import GPTQModel
model = GPTQModel.from_quantized("namgyu-youn/EXAONE-4.0-1.2B-GPTQ-W4A16-EoRA", device="cuda:0")

python

from vllm import LLM
llm = LLM(model="namgyu-youn/EXAONE-4.0-1.2B-GPTQ-W4A16-EoRA", dtype="float16")