Views
No views yet
RP 기반 튜닝 모델 제작데이터셋 정제상식 보완1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3device = "cuda" # the device to load the model onto
4
5model = AutoModelForCausalLM.from_pretrained("maywell/Synatra-7B-v0.3-RP")
6tokenizer = AutoTokenizer.from_pretrained("maywell/Synatra-7B-v0.3-RP")
7
8messages = [
9 {"role": "user", "content": "바나나는 원래 하얀색이야?"},
10]
11
12encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
13
14model_inputs = encodeds.to(device)
15model.to(device)
16
17generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
18decoded = tokenizer.batch_decode(generated_ids)
19print(decoded[0])| Metric | Value |
|---|---|
| Avg. | 57.38 |
| ARC (25-shot) | 62.2 |
| HellaSwag (10-shot) | 82.29 |
| MMLU (5-shot) | 60.8 |
| TruthfulQA (0-shot) | 52.64 |
| Winogrande (5-shot) | 76.48 |
| GSM8K (5-shot) | 21.15 |
| DROP (3-shot) | 46.06 |