Views
No views yet
1 arc arc/e boolq hswag obkqa piqa wino
2mxfp8 0.636,0.816,0.888
3
4Quant Perplexity Peak Memory Tokens/sec
5mxfp8 4.218 ± 0.027 16.02 GB 6871 arc arc/e boolq hswag obkqa piqa wino
2mxfp8 0.550,0.724,0.888,0.691,0.418,0.771,0.674
3qx86-hi 0.556,0.718,0.885,0.701,0.432,0.776,0.6841 arc arc/e boolq hswag obkqa piqa wino
2qx86-hi 0.642,0.819,0.895,0.716,0.454,0.785,0.6991models:
2 - model: nightmedia/Qwen3.5-9B-TNG-PKD-Qwopus-Coder
3 parameters:
4 weight: 1.6
5 - model: DavidAU/Qwen3.5-9B-The-Bradbury-F451-Pro-Writer-Uncensored-Heretic
6 parameters:
7 weight: 0.4
8merge_method: nuslerp
9dtype: bfloat16
10name: Qwen3.5-9B-TNG-PKD-Qwopus-Writer-Bradbury-F451pip install mlx-lm1from mlx_lm import load, generate
2
3model, tokenizer = load("Qwen3.5-9B-TNG-PKD-Qwopus-Writer-Bradbury-F451-mxfp8-mlx")
4
5prompt = "hello"
6
7if tokenizer.chat_template is not None:
8 messages = [{"role": "user", "content": prompt}]
9 prompt = tokenizer.apply_chat_template(
10 messages, add_generation_prompt=True, return_dict=False,
11 )
12
13response = generate(model, tokenizer, prompt=prompt, verbose=True)