Views
No views yet
| Model | Loss |
|---|---|
| Mistral-Hinglish-Instruct | 0.8 |
| Mistral-Instruct | 1.8 |
[INST] and [/INST] tokens. The very first instruction should begin with a begin of sentence id. The next instructions should not. The assistant generation will be ended by the end-of-sentence token id.text = "<s>[INST] What is your favourite condiment? [/INST]"
"Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> "
"[INST] Do you have mayonnaise recipes? [/INST]"apply_chat_template() method:1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3device = "cuda" # the device to load the model onto
4
5model = AutoModelForCausalLM.from_pretrained("arshadshk/Mistral-Hinglish-7B-Instruct-v0.2")
6tokenizer = AutoTokenizer.from_pretrained("arshadshk/Mistral-Hinglish-7B-Instruct-v0.2")
7
8messages = [
9 {"role": "user", "content": "What is your favourite condiment?"},
10 {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
11 {"role": "user", "content": "Do you have mayonnaise recipes?"}
12]
13
14encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
15
16model_inputs = encodeds.to(device)
17model.to(device)
18
19generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
20decoded = tokenizer.batch_decode(generated_ids)
21print(decoded[0])