After your question add two new lines followed by Chip:
(\n\nChip:)
Example code snippet to run on your own system:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
chip_map= {'gpt_neox.embed_in': 0,
'gpt_neox.layers': 0,
'gpt_neox.final_layer_norm': 0,
'embed_out': 0}
name = "Rallio67/chip_12B_instruct_alpha"
model = AutoModelForCausalLM.from_pretrained(name, device_map=chip_map, torch_dtype=torch.float16, load_in_8bit=True )
tokenizer = AutoTokenizer.from_pretrained(name)
def generate_from_model(model, tokenizer):
encoded_input = tokenizer(text, return_tensors='pt')
output_sequences = model.generate(
input_ids=encoded_input['input_ids'].cuda(0),
do_sample=True,
max_new_tokens=35,
num_return_sequences=1,
top_p=0.95,
temperature=0.5,
penalty_alpha=0.6,
top_k=4,
output_scores=True,
return_dict_in_generate=True,
repetition_penalty=1.03,
eos_token_id=0,
use_cache=True
)
gen_sequences = output_sequences.sequences[:, encoded_input['input_ids'].shape[-1]:]
for sequence in gen_sequences:
new_line=tokenizer.decode(sequence, skip_special_tokens=True)
print(new_line)
text = "User: Why is everyone so excited about AI chatbots these days?"
generate_from_model(model,tokenizer)
#Chip: Chatbots are a great way to automate simple tasks. They can help you save time and make your life easier. For example, they can
User: Are a manatee and a dolphin more likely to be friends or foes?
Chip: A manatee and a dolphin are both mammals, so they are more likely to be friends than foes.
Chip: I’m not sure. I’d have to look into it.
Chip: I’d say they’re more likely to be enemies. A manatee is a mammal, and a dolphin is a mammal, so they’re more likely to be enemies than friends.
Chip: A manatee and a dolphin are more likely to be friends. Manatees and dolphins are both marine mammals that live in the ocean. They have similar habitats and lifestyles, and they are both herbivores. They also have similar diets, so they can eat the same food.
User: Why are roses red?
Chip: Roses are red because they are made up of a pigment called anthocyanin. Anthocyanin is a pigment that gives the plant its color. It is also found in other flowers, such as daisies, tulips, and violets.
Chip: Roses are red because they contain a pigment called anthocyanin. Anthocyanins are a type of flavonoid that gives plants their color. Flavonoids are a class of compounds found in plants and other organisms.
Chip: Roses are red because they are a member of the rose family. The rose family is a large group of plants that includes many different species. The most common type of rose is the garden rose, which is a hybrid of two species of roses. The two species are Rosa gallica and Rosa rugosa. The rose family is named after the genus name of the first species, Rosa.
Chip: Roses are red because they have a gene that makes them red. The gene is called the anthocyanin gene, and it is found in most plants. It is also found in some other animals, such as birds, lizards, and insects. The anthocyanin gene is responsible for the color of the flower.
User: What's a good recipe for bbq ribs?
Chip: Ingredients:
• 4 racks of baby back ribs
• 1 bottle of your favorite BBQ sauce
• 1/2 cup of brown sugar
• 1/2 cup of honey
• 1/4 cup of apple cider vinegar
• 2 cloves of garlic, minced
• 2 tablespoons of chili powder
• 1 teaspoon of paprika
• 1/2 teaspoon of cayenne pepper
• 1/4 teaspoon of black pepper
• 1/4 teaspoon of salt
Instructions:
Preheat oven to 350 degrees F (175 degrees C).
Place the ribs in a large roasting pan and cover with the BBQ sauce.
In a small bowl, mix together the brown sugar, honey, vinegar, garlic, chili powder, paprika, cayenne pepper, black pepper, and salt.
Pour the mixture over the ribs and rub it into the meat.
Cover
Chip: Ingredients:
• 1 rack of baby back ribs
• 1 bottle of your favorite bbq sauce
• 1 onion, sliced
• 2 cloves of garlic, minced
• 1 teaspoon of paprika
• 1 teaspoon of cumin
• 1 teaspoon of chili powder
• 1 teaspoon of oregano
• 1 teaspoon of salt
• 1 teaspoon of pepper
• 1/2 cup of white vinegar
Instructions:
Preheat your oven to 350 degrees Fahrenheit.
Place the ribs in a baking dish and cover with the bbq sauce.
In a medium bowl, combine the onion, garlic, paprika, cumin, chili powder, oregano, salt, and pepper.
Rub the mixture all over the ribs.
Bake in the preheated oven for 2 hours or until the ribs are tender.
Remove from the oven and let cool for 10 minutes
Chip:
Preheat oven to 350 degrees F (175 degrees C).
Place the ribs in a baking dish and cover with barbecue sauce.
Bake in preheated oven for 1 hour or until the ribs are tender.
Remove from oven and let cool.
Enjoy!
Chip:
Preheat oven to 350 degrees F (175 degrees C).
Place the ribs in a roasting pan, and season with your favorite barbecue rub.
Roast the ribs in the preheated oven for 2 hours or until the ribs are tender and cooked through.
Serve the ribs with your favorite barbecue sauce.
User: Where does Santa Claus live?
Chip: Santa Claus lives in the North Pole. He has a house there, and he spends most of his time there.
Chip: Santa Claus lives in the North Pole. He has a workshop in the North Pole, and he sends presents to children all over the world.
Chip: Santa Claus lives in the North Pole. He is a jolly old man who delivers presents to all the good boys and girls on Christmas Eve.
Chip: Santa Claus lives in the North Pole. He has a very large house there, and he has a lot of elves who help him.
