Views
No views yet
| Property | Value |
|---|---|
| Quantization | 8-bit affine |
| Bits per weight | 8.500 |
| Group size | 64 |
| Size | 36.8 GB |
1pip install mlx-lm
2mlx_lm.generate --model nicolasembleton/Apodex-1.1-mini-MLX-8bit --prompt "Hello"1from mlx_lm import load, generate
2
3model, tokenizer = load("nicolasembleton/Apodex-1.1-mini-MLX-8bit")
4
5prompt = "Hello"
6
7if tokenizer.chat_template is not None:
8 messages = [{"role": "user", "content": prompt}]
9 prompt = tokenizer.apply_chat_template(
10 messages, add_generation_prompt=True
11 )
12
13response = generate(model, tokenizer, prompt=prompt, verbose=True)