Views
No views yet
mlx_lm.convert with 4-bit quantization (q_bits=4, q_group_size=64)1from mlx_lm import load, generate
2
3model, tokenizer = load("rafal-adamczyk/Qwen3.5-9B-MLX-4bit")
4response = generate(model, tokenizer, prompt="Hello!", verbose=True)