Views
No views yet
1 arc arc/e boolq hswag obkqa piqa wino
2mxfp4 0.646,0.838,0.902,0.778,0.444,0.822,0.703
3Text only
4mxfp4 0.657,0.862,0.906,0.766,0.490,0.825,0.692
5
6Quant Perplexity Peak Memory Tokens/sec
7mxfp4 5.286 ± 0.038 25.33 GB1 arc arc/e boolq hswag obkqa piqa wino
2qx64-hi 0.644,0.818,0.909
3mxfp4 0.626,0.813,0.901
4
5Quant Perplexity Peak Memory Tokens/sec
6qx64-hi 3.954 ± 0.025 32.86 GB 1311
7mxfp4 4.170 ± 0.028 25.33 GB 15991 arc arc/e boolq hswag obkqa piqa wino
2qx64-hi 0.647,0.835,0.909
3mxfp4 0.626,0.813,0.901
4
5Quant Perplexity Peak Memory Tokens/sec
6mxfp8 4.012 ± 0.026 42.65 GB 1543
7qx64-hi 3.973 ± 0.026 32.86 GB 1532
8mxfp4 4.170 ± 0.028 25.33 GB 1471preserve_thinking flag.pip install mlx-lm1from mlx_lm import load, generate
2
3model, tokenizer = load("SuperQwen-AgentWorld-35B-A3B-abliterated-mxfp4-mlx")
4
5prompt = "hello"
6
7if tokenizer.chat_template is not None:
8 messages = [{"role": "user", "content": prompt}]
9 prompt = tokenizer.apply_chat_template(
10 messages, add_generation_prompt=True, return_dict=False,
11 )
12
13response = generate(model, tokenizer, prompt=prompt, verbose=True)