gemma-2-9b-gptqmodel-4bit – AI Model by iproskurina | AlphaNeural AI

You can deploy this model and start earning money today!

iproskurina/
gemma-2-9b-gptqmodel-4bit

like

0

text-generation

text-generation-inference

Views

No views yet

gemma-2-9b - GPTQ (4-bit)

Source model: google/gemma-2-9b

This model was quantized to 4-bit using GPTQModel.

Quantization parameters:

bits: 4
group_size: 128
damp_percent: 0.05
desc_act: False

Usage

            # pip install transformers gptqmodel --no-build-isolation
            from gptqmodel import GPTQModel
            model_id = "iproskurina/gemma-2-9b-gptqmodel-4bit"
            model = GPTQModel.load(model_id)
            result = model.generate("Uncovering deep insights")[0]
            print(model.tokenizer.decode(result))