Views
No views yet
1 arc arc/e boolq hswag obkqa piqa wino
2mxfp8 0.499,0.698,0.807,0.680,0.408,0.764,0.609
3
4Quant Perplexity Peak Memory Tokens/sec
5mxfp8 9.452 ± 0.079 10.87 GB 32501 arc arc/e boolq hswag obkqa piqa wino
2q8-hi 0.529,0.744,0.745,0.658,0.412,0.760,0.5971 arc arc/e boolq hswag obkqa piqa wino
2mxfp8 0.467,0.632,0.779,0.690,0.412,0.745,0.6171 arc arc/e boolq hswag obkqa piqa wino
2bf16 0.464,0.583,0.826,0.624,0.398,0.717,0.575
3mxfp8 0.460,0.575,0.829,0.624,0.394,0.711,0.567
4mxfp4 0.441,0.582,0.816,0.615,0.406,0.708,0.5521models:
2 - model: DavidAU/LFM2-8B-A1B-GLM-4.7-Flash-Thinking-Quantum-IQ1C-P
3 parameters:
4 weight: 1.4
5 - model: LFM2-8B-A1B-The-Deckard-Series-C-Uncensored-Heretic-A1
6 parameters:
7 weight: 0.6
8merge_method: nuslerp
9dtype: bfloat16
10name: LFM2-8B-A1B-GLM-Deckardpip install mlx-lm1from mlx_lm import load, generate
2
3model, tokenizer = load("LFM2-8B-A1B-GLM-Deckard-mxfp8-mlx")
4
5prompt = "hello"
6
7if tokenizer.chat_template is not None:
8 messages = [{"role": "user", "content": prompt}]
9 prompt = tokenizer.apply_chat_template(
10 messages, add_generation_prompt=True, return_dict=False,
11 )
12
13response = generate(model, tokenizer, prompt=prompt, verbose=True)