Views
No views yet
1!pip install transformers
2!pip install peft
3!pip install -U bitsandbytes1from transformers import AutoTokenizer, AutoModelForCausalLM
2tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
3model = AutoModelForCausalLM.from_pretrained("SwastikM/Meta-Llama-3-8B-Instruct_bitsandbytes_4bit")
4
5messages = [
6 {"role": "system", "content": "You are a Coder."},
7 {"role": "user", "content": "How to ctrate a list in Python?"}
8]
9
10input_ids = tokenizer.apply_chat_template(
11 messages,
12 add_generation_prompt=True,
13 return_tensors="pt"
14).to(model.device)
15
16terminators = [
17 tokenizer.eos_token_id,
18 tokenizer.convert_tokens_to_ids("<|eot_id|>")
19]
20
21outputs = model.generate(
22 input_ids,
23 max_new_tokens=256,
24 eos_token_id=terminators,
25 do_sample=False,
26 temperature=0.0
27)
28
29response = outputs[0][input_ids.shape[-1]:]
30print(tokenizer.decode(response, skip_special_tokens=True))In Python, you can create a list in several ways:
1. Using the `list()` function:
my_list = list()
This creates an empty list.
2. Using square brackets `[]`:
my_list = []
This also creates an empty list.
3. Using the `list()` function with an iterable (such as a string or a tuple):
my_list = list("hello")
print(my_list) # Output: ['h', 'e', 'l', 'l', 'o']
4. Using the `list()` function with a range of numbers:
my_list = list(range(1, 6))
print(my_list) # Output: [1, 2, 3, 4, 5]
5. Using the `list()` function with a dictionary:
my_dict = {"a": 1, "b": 2, "c": 3}
my_list = list(my_dict.keys())
print(my_list) # Output: ['a', 'b', 'c']
Note that in Python, lists are mutable, meaning you can add, remove, or modify elements after creating the list.| Model | Total Size |
|---|---|
| Base Model | 28 GB |
| 4bitQuantized | 5.21 GB |