Views
No views yet
transformers, einops1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_id = "crumb/gpt2a-pile-test-285m"
4model = AutoModelForCausalLM.from_pretrained(
5 model_id,
6 trust_remote_code = True,
7 device_map="auto"
8)
9tokenizer = AutoTokenizer.from_pretrained(model_id)
10
11input_ids = tokenizer("In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.", return_tensors="pt").input_ids.cuda()
12outputs = model.generate(input_ids, max_new_tokens=128, temperature=0.7, do_sample=True, top_p=0.95, repetition_penalty=1.1)
13print(tokenizer.batch_decode(outputs)[0])
14"""
15<s> In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English. It turns out that only a few people believed their father’s name had been made public by their mother and her father.
16
17As the study finds, scientists discovered the secretive organic matter in the Andes mountains: “In a forest surrounded by a stream of lakes, an unseen rock formed with a series of tuberous orbits.”
18
19They found that the mysterious bodies were buried in some parts of the mountain, known as the Andes mountain. The researchers then searched for the body, which they identified with the Butterfly.
20
21The discovery of the body is the result of
22"""| param calculation | params |
|---|---|
| model | 809,579,521 |
| model - model.transformer.wte | 539,045,889 |
| model - model.transformer.wte[0] (llama2-70b embeddings without projection) | 547,435,521 |
| model - model.transformer.wte - model.lm_head | 268,505,089 |
| model - model.transformer.wte[0] - model.lm_head[1] (minus all params taken from llama2-70b) | 285,291,521 |