Views
No views yet
microsoft/phi-2.
Refer to the original model card for more details on the model.1pip install mlx
2git clone https://github.com/ml-explore/mlx-examples.git
3cd mlx-examples/llms/hf_llm
4python generate.py --model marcel/phi-2-openhermes-30k --prompt "My name is"1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained(
4 "marcel/phi-2-openhermes-30k",
5 low_cpu_mem_usage=True,
6 device_map="auto",
7 trust_remote_code=True,
8 torch_dtype=torch.float16,
9)
10tokenizer = AutoTokenizer.from_pretrained("phi-2-openhermes-30k")
11
12input_text = "### Human: Give me a good recipe for a chinese dish\n\n### Assistant:"
13
14outputs = model.generate(
15 tokenizer(input_text, return_tensors="pt").to(model.device)['input_ids'],
16 max_length=1024,
17 temperature=0.7,
18 top_p=0.9,
19 do_sample=True,
20 pad_token_id=tokenizer.pad_token_id,
21 eos_token_id=tokenizer.eos_token_id,
22)
23print(tokenizer.decode(outputs[0], skip_special_tokens=True))| Metric | Value |
|---|---|
| Avg. | 60.37 |
| AI2 Reasoning Challenge (25-Shot) | 61.01 |
| HellaSwag (10-Shot) | 74.72 |
| MMLU (5-Shot) | 57.17 |
| TruthfulQA (0-shot) | 45.38 |
| Winogrande (5-shot) | 74.90 |
| GSM8k (5-shot) | 49.05 |