Views
No views yet
### Instruction:
{query}
### Response:
<Leave new line for model to respond> 1from transformers import AutoTokenizer, AutoModelForCausalLM,pipeline
2
3tokenizer = AutoTokenizer.from_pretrained("mwitiderrick/open_llama_3b_instruct_v_0.2")
4model = AutoModelForCausalLM.from_pretrained("mwitiderrick/open_llama_3b_instruct_v_0.2")
5query = "Provide step-by-step instructions for making a sweet chicken bugger"
6text_gen = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=500)
7output = text_gen(f"### Instruction:\n{query}\n### Response:\n")
8print(output[0]['generated_text'])
9"""
10### Instruction:
11Provide step-by-step instructions for making a sweet chicken bugger
12### Response:
13Step 1: Gather your ingredients
141. 1/2 cup of sugar
152. 1/2 cup of corn syrup
163. 1/2 cup of water
174. 1/2 cup of vegetable oil
185. 1/2 cup of vanilla extract
196. 1/2 cup of baking soda
207. 1/2 cup of salt
218. 1/2 cup of flour
229. 1/2 cup of milk
2310. 1/2 cup of egg whites
24
25Step 2: Mix the ingredients together
261. Combine the sugar, corn syrup, water, vegetable oil, vanilla extract, baking soda, and salt in a large bowl.
272. Whisk together until smooth.
283. Add the flour and mix until combined.
294. Add the milk and egg whites and mix until combined.
305. Pour the mixture into a greased 9x13 inch baking pan.
316. Bake for 30 minutes or until a toothpick inserted into the center comes out clean.
32
33Step 3: Make the chicken bugger
341. Preheat the oven to 350 degrees Fahrenheit.
352. In a large bowl, combine the corn syrup, sugar, and cornstarch.
363. Add the chicken and mix well.
374. Divide the mixture into 12 equal portions and shape each portion into a chicken shape.
385. Place the chicken shapes on a baking sheet lined with parchment paper.
396. Bake for 15 minutes or until the chicken is cooked through.
407. Remove the chicken from the oven and allow to cool for 5 minutes.
418. Using a fork, carefully remove the chicken from the shells and place on a serving platter.
429. Serve with a side of gravy.
43
44Step 4: Make the gravy
451. In a small saucepan, combine the cornstarch and water.
462. Stir until the mixture is smooth and begins to thicken.
473. Add the chicken broth and bring to a boil.
484. Reduce the heat to low and simmer for 10 minutes or until the gravy is
49"""
| Groups |Version|Filter|n-shot| Metric | Value | |Stderr|
|----------|-------|------|-----:|-----------|-------:|---|-----:|
|truthfulqa|N/A |none | 0|acc | 0.3166|± |0.0012|
| | |none | 0|bleu_max | 23.7766|± |0.7660|
| | |none | 0|bleu_acc | 0.3207|± |0.0163|
| | |none | 0|bleu_diff | -7.1853|± |0.7396|
| | |none | 0|rouge1_max | 48.6534|± |0.8706|
| | |none | 0|rouge1_acc | 0.2766|± |0.0157|
| | |none | 0|rouge1_diff| -9.8011|± |0.7883|
| | |none | 0|rouge2_max | 31.9289|± |0.9637|
| | |none | 0|rouge2_acc | 0.2399|± |0.0149|
| | |none | 0|rouge2_diff|-11.3958|± |0.9220|
| | |none | 0|rougeL_max | 45.4592|± |0.8754|
| | |none | 0|rougeL_acc | 0.2754|± |0.0156|
| | |none | 0|rougeL_diff|-10.0740|± |0.7807|| Metric | Value |
|---|---|
| Avg. | 38.97 |
| AI2 Reasoning Challenge (25-Shot) | 38.48 |
| HellaSwag (10-Shot) | 66.77 |
| MMLU (5-Shot) | 25.34 |
| TruthfulQA (0-shot) | 38.16 |
| Winogrande (5-shot) | 63.46 |
| GSM8k (5-shot) | 1.59 |