Views
No views yet
1 arc arc/e
2qx85 0.456,0.519,0.622,0.704,0.392,0.774,0.680
3qx64-hi 0.445,0.513,0.622,0.704,0.384,0.786,0.702
4mxfp4 0.455,0.510,...
5
6Quant Perplexity Speed(t/s) Memory
7qx85 3.762 ± 0.024 448 98.86 GB
8qx64-hi 3.778 ± 0.024 435 98.28 GB
9mxfp4 3.909 ± 0.025 535 71.94 GB
10pip install mlx-lm1from mlx_lm import load, generate
2
3model, tokenizer = load("Qwen3.5-122B-A10B-Text-qx85-mlx")
4
5prompt = "hello"
6
7if tokenizer.chat_template is not None:
8 messages = [{"role": "user", "content": prompt}]
9 prompt = tokenizer.apply_chat_template(
10 messages, add_generation_prompt=True, return_dict=False,
11 )
12
13response = generate(model, tokenizer, prompt=prompt, verbose=True)