starcoder2-15b-w4-autoawq-gemm – AI Model by shavera | AlphaNeural AI

You can deploy this model and start earning money today!

shavera/
starcoder2-15b-w4-autoawq-gemm

like

0

bigcode/starcoder2-15b

bigcode-openrail-m

Views

No views yet

Model Card for Model ID

This is a int4_awq quantized checkpoint of bigcode/starcoder2-15b. It takes about 10GB of VRAM.

Running this Model

Run via docker with text-generation-interface:

docker run --gpus all --shm-size 64g -p 8080:80 -v ~/.cache/huggingface:/data \
    ghcr.io/huggingface/text-generation-inference:3.1.0 \
	--model-id shavera/starcoder2-15b-w4-autoawq-gemm