Views
No views yet
1q6 0.594,0.746,0.881,0.779,0.464,0.816,0.751
2q8 0.596,0.748,0.881,0.779,0.458,0.819,0.7511regular 0.590,0.742,0.883,0.781,0.458,0.822,0.751
2heretic 0.596,0.748,0.881,0.779,0.458,0.819,0.7511gemma-3-27b-it-heretic
2q8 0.557,0.711,0.868,0.533,0.452,0.706,0.695
3
4Gemma-3-27b-it-HERETIC-Gemini-Deep-Reasoning
5q8 0.596,0.748,0.881,0.779,0.458,0.819,0.751 pip install mlx-lm1from mlx_lm import load, generate
2
3model, tokenizer = load("Gemma-3-27b-it-HERETIC-Gemini-Deep-Reasoning-q6-mlx")
4
5prompt = "hello"
6
7if tokenizer.chat_template is not None:
8 messages = [{"role": "user", "content": prompt}]
9 prompt = tokenizer.apply_chat_template(
10 messages, add_generation_prompt=True, return_dict=False,
11 )
12
13response = generate(model, tokenizer, prompt=prompt, verbose=True)