You can deploy this model and start earning money today!

cyboghostginx/
Llama3.1-8B-mlx-4Bit

Name: Llama3.1-8B-mlx-4Bit
Availability: InStock

Llama3.1-8B-mlx-4Bit

Built with Llama. 4-bit MLX conversion of Meta's Llama 3.1 8B base model for Apple silicon. 4.5 GB, single shard, converted with mlx-lm 0.26.4 from cyboghostginx/Llama3.1-8B. Weights only, no fine-tuning.

This is the base model, not Instruct. It ships no chat template, so it completes text rather than answering turns. For chat, quantize an Instruct checkpoint instead.

bash

pip install mlx-lm
mlx_lm.generate --model cyboghostginx/Llama3.1-8B-mlx-4Bit \
                --prompt "The three laws of robotics are" --max-tokens 256

python

from mlx_lm import load, generate

model, tokenizer = load("cyboghostginx/Llama3.1-8B-mlx-4Bit")
print(generate(model, tokenizer, prompt="The three laws of robotics are", verbose=True))

Higher precision: 8-bit MLX, 8.5 GB.

cyboghostginx/Llama3.1-8B-mlx-4Bit

Llama3.1-8B-mlx-4Bit

License

cyboghostginx/
Llama3.1-8B-mlx-4Bit