You can deploy this model and start earning money today!

cyboghostginx/
Dolphin-X1-8B-mlx-4Bit

Name: Dolphin-X1-8B-mlx-4Bit
Availability: InStock

text-generation-inference

endpoints_compatible

4-bit

Views

No views yet

Dolphin-X1-8B-mlx-4Bit

4-bit MLX conversion of dphn/Dolphin-X1-8B for Apple silicon. 4.5 GB, single shard, converted with mlx-lm 0.26.4. Weights only, no fine-tuning.

bash

pip install mlx-lm
mlx_lm.generate --model cyboghostginx/Dolphin-X1-8B-mlx-4Bit \
                --prompt "your prompt" --max-tokens 512

python

from mlx_lm import load, generate

model, tokenizer = load("cyboghostginx/Dolphin-X1-8B-mlx-4Bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "hello"}], tokenize=False, add_generation_prompt=True
)
print(generate(model, tokenizer, prompt=prompt, verbose=True))

License

Built with Llama. Governed by the Llama 3.1 Community License, inherited through the base model. Safety behaviour follows the Dolphin base, so it answers requests a stock instruct model declines. You are responsible for how you deploy it.