4-bit MLX conversion of dphn/Dolphin-Mistral-24B-Venice-Edition for Apple silicon. 13.3 GB across 3 shards, converted with mlx-lm 0.26.4. Weights only, no fine-tuning.

bash

pip install mlx-lm
mlx_lm.generate --model cyboghostginx/Dolphin-Mistral-24B-Venice-Edition-mlx-4Bit \
                --prompt "your prompt" --max-tokens 512

python

from mlx_lm import load, generate

model, tokenizer = load("cyboghostginx/Dolphin-Mistral-24B-Venice-Edition-mlx-4Bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "hello"}], tokenize=False, add_generation_prompt=True
)
print(generate(model, tokenizer, prompt=prompt, verbose=True))

License

Apache-2.0, inherited from the base model. Safety behaviour follows the Dolphin base, so it answers requests a stock instruct model declines. You are responsible for how you deploy it.