A 4-bit MLX quantization of
empero-ai/Qwythos-9B-Claude-Mythos-5-1M, published as
Qwen-9B-Claude-Fable-5-1M-MLX-4bit and optimized for Apple Silicon with a minimal memory footprint. LM Studio compatibilty fixed, please update.
Remove sfw if you don't have sfw from
Socket installed. It's free and an improvement over running pip or npm raw.
1from mlx_lm import load, generate
2
3model, tokenizer = load("shamsghi/Qwen-9B-Claude-Fable-5-1M-MLX-4bit")
4response = generate(model, tokenizer, prompt="Hello!", max_tokens=256)
5print(response)
See
base model card for evaluation results. 4-bit quantization may introduce minor quality degradation vs. 8-bit.
Apache 2.0 — same as the base model.
