You can deploy this model and start earning money today!

b4rtaz/
Llama-3_1-8B-Q40-Instruct-Distributed-Llama

Name: Llama-3_1-8B-Q40-Instruct-Distributed-Llama
Availability: InStock

Views

No views yet

This is converted Llama 3.1 8B Instruct model to Distributed Llama format. The model is quantized to Q40.

🚀 How to Run?

make dllama

./dllama chat --model dllama_model_llama3.1_instruct_q40.m --tokenizer dllama_tokenizer_llama_3_1.t --buffer-float-type q80 --nthreads 4