Views
No views yet
mlx-lm version 0.26.0. It ran for 15 steps with a batch size of 16, completing in a few seconds on a MacBook Pro M1 Pro (8-core CPU, 16GB RAM). Peak memory usage was 8.3GB. The dataset contained 225 low-quality training pairs (240 lines trained total)./no_think and the following generation parameters:--temp 0.7--top-p 0.8--top-k 20--min-p 0pip install mlx-lm1from mlx_lm import load, generate
2
3model, tokenizer = load("DuckyBlender/diegogpt-v2-mlx-bf16")
4
5prompt = "are you red hat hacker?"
6
7if tokenizer.chat_template is not None:
8 messages = [
9 {"role": "user", "content": user_input}
10 ]
11 prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False, enable_thinking=False)
12else:
13 prompt = user_input
14
15sampler = make_sampler(temp=0.7, top_p=0.8, top_k=20, min_p=0)
16
17response = mlx_lm.generate(
18 model,
19 tokenizer,
20 prompt=prompt,
21 sampler=sampler,
22 verbose=True
23)1mlx_lm.generate \
2 --model "DuckyBlender/diegogpt-v2-mlx-bf16" \
3 --temp 0.7 \
4 --top-p 0.8 \
5 --top-k 20 \
6 --min-p 0 \
7 --system "/no_think" \
8 --prompt "are you red hat hacker?"