Views
No views yet
<s>[INST] {sys_prompt} {instruction} [/INST] {sys_prompt}
### Instruction:
{instruction}
### Response:
1from transformers import AutoTokenizer
2import transformers
3import torch
4
5model = "lodrick-the-lafted/Grafted-Hermetic-Platypus-A-2x7B"
6
7tokenizer = AutoTokenizer.from_pretrained(model)
8pipeline = transformers.pipeline(
9 "text-generation",
10 model=model,
11 model_kwargs={"torch_dtype": torch.bfloat16},
12)
13
14messages = [{"role": "user", "content": "Give me a cooking recipe for an orange pie."}]
15prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
16outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.95)
17print(outputs[0]["generated_text"])| Metric | Value |
|---|---|
| Avg. | 64.65 |
| AI2 Reasoning Challenge (25-Shot) | 59.47 |
| HellaSwag (10-Shot) | 82.95 |
| MMLU (5-Shot) | 62.15 |
| TruthfulQA (0-shot) | 61.49 |
| Winogrande (5-shot) | 77.43 |
| GSM8k (5-shot) | 44.43 |