Views
No views yet
1pip install mlx-lm
2mlx_lm.generate --model darthcrawl/artifex-rp-orpheus-llama-3.1-8b-mlx-6bit \
3 --system-prompt "Write in vivid, restrained, intelligent prose. Concrete sensory detail. Never pretentious. Tension carries more weight than description. Match the user's energy and length." \
4 --prompt "She hadn't seen him in three years." \
5 --temp 0.85 --top-p 0.95 --max-tokens 400| Parameter | Value |
|---|---|
| Bits | 6 |
| Group size | 32 |
| Effective bits/weight | 7.0 |
| Size | 6.6GB |
| Repo | Size | Fits on |
|---|---|---|
| artifex-rp-orpheus-llama-3.1-8b | ~16GB | 24GB+ Mac (BF16) |
| artifex-rp-orpheus-llama-3.1-8b-mlx-8bit | 8.4GB | 16GB Mac |
| artifex-rp-orpheus-llama-3.1-8b-mlx-6bit | 6.6GB | 16GB Mac |
| artifex-rp-orpheus-llama-3.1-8b-mlx-4bit-dwq | 4.7GB | 8GB Mac |