Views
No views yet
1# Format prompt
2message = [
3 {"role": "system", "content": "You are a helpful assistant chatbot."},
4 {"role": "user", "content": "What is a Large Language Model?"}
5]
6tokenizer = AutoTokenizer.from_pretrained(new_model)
7prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)
8
9# Create pipeline
10pipeline = transformers.pipeline(
11 "text-generation",
12 model=new_model,
13 tokenizer=tokenizer
14)
15
16# Generate text
17sequences = pipeline(
18 prompt,
19 do_sample=True,
20 temperature=0.7,
21 top_p=0.9,
22 num_return_sequences=1,
23 max_length=200,
24)
25print(sequences[0]['generated_text'])
26
27# <s>[INST] <<SYS>>
28# You are a helpful assistant chatbot.
29# <</SYS>>
30#
31# What is a Large Language Model? [/INST]
32# <LANG-LMT>
33# Largely, it is a machine learning model that is trained on a large dataset and is capable of generating large amounts of text with a certain degree of accuracy.
34#
35# A: If you are talking about a computer program that can generate texts, you can look at the topic of Natural Language Generation (NLG) for a more precise definition.
36# The main difference between NLG and machine learning is that NLG is a subfield of AI and is used to generate text from an input, while machine learning is used to analyze data, make predictions and classify it.
37| Tasks | Metric | Value | Stderr | |
|---|---|---|---|---|
| arc_challenge | acc | 0.2807 | ± | 0.0131 |
| acc_norm | 0.3106 | ± | 0.0135 | |
| arc_easy | acc | 0.6107 | ± | 0.0100 |
| acc_norm | 0.5547 | ± | 0.0102 | |
| boolq | acc | 0.5865 | ± | 0.0086 |
| hellaswag | acc | 0.4478 | ± | 0.0050 |
| acc_norm | 0.5924 | ± | 0.0049 | |
| openbookqa | acc | 0.2160 | ± | 0.0184 |
| acc_norm | 0.3600 | ± | 0.0215 | |
| piqa | acc | 0.7280 | ± | 0.0104 |
| acc_norm | 0.7301 | ± | 0.0104 | |
| winogrande | acc | 0.5856 | ± | 0.0138 |