You can deploy this model and start earning money today!

Zenabius/
calme-3.2-instruct-78b-exl2-6.5bpw

Name: calme-3.2-instruct-78b-exl2-6.5bpw
Availability: InStock

safetensors

qwen2

MaziyarPanahi/calme-3.2-instruct-78b

quantized

6-bit

exl2

Views

No views yet

Bits - 6.5bpw

EXL2 Quantizations of calme-3.2-instruct-78b

Using exllamav2 release 0.2.6 for quantization.

Original model: https://huggingface.co/MaziyarPanahi/calme-3.2-instruct-78b

"quantization_config": {
    "quant_method": "exl2",
    "version": "0.2.6",
    "bits": 6.5,
    "head_bits": 8,
    "calibration": {
        "rows": 115,
        "length": 2048,
        "dataset": "(default)"
    }

Zenabius/calme-3.2-instruct-78b-exl2-6.5bpw

Zenabius/
calme-3.2-instruct-78b-exl2-6.5bpw