Views
No views yet

ChatML prompt template:<|im_start|>system
{System}<|im_end|>
<|im_start|>user
{User}<|im_end|>
<|im_start|>assistant
{Assistant}1
2from transformers import AutoModelForCausalLM, AutoTokenizer
3import torch
4
5model_name = "Tsunami-th/Tsunami-0.5x-7B-Instruct"
6
7model = AutoModelForCausalLM.from_pretrained(
8 model_name,
9 torch_dtype="auto",
10 device_map="auto"
11)
12tokenizer = AutoTokenizer.from_pretrained(model_name)
13
14messages = [
15 {"role": "system", "content": "You are a helpful assistant."},
16 {"role": "user", "content": "สวัสดีครับ"}
17]
18text = tokenizer.apply_chat_template(
19 messages,
20 tokenize=False,
21 add_generation_prompt=True
22)
23
24inputs = tokenizer(text, return_tensors="pt")
25inputs = inputs.to(model.device)
26with torch.no_grad():
27 output = model.generate(**inputs, max_new_tokens=512)
28
29response = tokenizer.decode(output[0, len(inputs['input_ids'][0]):], skip_special_tokens=True)| Metric | Value |
|---|---|
| Avg. | 29.80 |
| IFEval (0-Shot) | 70.99 |
| BBH (3-Shot) | 37.36 |
| MATH Lvl 5 (4-Shot) | 4.83 |
| GPQA (0-shot) | 8.61 |
| MuSR (0-shot) | 18.57 |
| MMLU-PRO (5-shot) | 38.42 |