You can deploy this model and start earning money today!

minicreeper/
Mistral-Small-24B-Instruct-2501-bnb-4bit

Name: Mistral-Small-24B-Instruct-2501-bnb-4bit
Availability: InStock

Views

No views yet

We quantified mistralai/Mistral-Small-24B-Instruct-2501 to 4bit model using BitsAndBytes.

To use this model you need install BitsAndBytes at first:

pip install -U bitsandbytes

Then, use AutoModelForCausalLM:

from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("minicreeper/Mistral-Small-24B-Instruct-2501-bnb-4bit")