Views
No views yet
Gemma-3-27b-it-HERETIC-Gemini-Deep-Reasoning
q8 0.596,0.748,0.881,0.779,0.458,0.819,0.751
Gemma3-27B-it-vl-HERETIC-GLM4.7-2200x-cp515
q8 0.566,0.737,0.876,0.745,0.420,0.805,0.759pip install mlx-lm1from mlx_lm import load, generate
2
3model, tokenizer = load("Gemma3-27B-it-vl-HERETIC-GLM4.7-2200x-cp515-q8-mlx")
4
5prompt = "hello"
6
7if tokenizer.chat_template is not None:
8 messages = [{"role": "user", "content": prompt}]
9 prompt = tokenizer.apply_chat_template(
10 messages, add_generation_prompt=True, return_dict=False,
11 )
12
13response = generate(model, tokenizer, prompt=prompt, verbose=True)