Views
No views yet
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_path = "WiNGPT-Llama-3-8B-Chat"
4device = "cuda"
5
6tokenizer = AutoTokenizer.from_pretrained(model_path)
7model = AutoModelForCausalLM.from_pretrained(model_path).to(device)
8model = model.eval()
9
10
11text = 'User：WiNGPT, 你好<|end_of_text|>\n Assistant：'
12inputs = tokenizer.encode(text, return_tensors="pt").to(device)
13outputs = model.generate(inputs, repetition_penalty=1.1, max_new_tokens=1024)
14response = tokenizer.decode(outputs[0])
15print(response)
16
17## 输出结果：你好！今天我能为你做些什么？<|end_of_text|>"{% for message in messages %}{% if message['role'] == 'system' %}System：{% endif %}{% if message['role'] == 'user' %}User：{% endif %}{% if message['role'] == 'assistant' %}Assistant：{% endif %}{{ message['content'] }}<|end_of_text|>\n {% endfor %}Assistant："User：WiNGPT, 你好<|end_of_text|>\n Assistant：User：WiNGPT, 你好<|end_of_text|>\n Assistant：你好！今天我能为你做些什么？<|end_of_text|>\n User：你是谁？<|end_of_text|>\n Assistant：System：作为医疗领域的智能助手，WiNGPT将提供中英翻译服务。用户输入的中文或英文内容将由WiNGPT进行准确的翻译，以满足用户的语言需求。<|end_of_text|>\n User：Life is short, you know, and time is so swift; Rivers are wide, so wide, and ships sail far.<|end_of_text|>\n Assistant：| 名称 | 训练策略 | 长度 | 精度 | 学习率 | Weight_decay | Epochs | GPUs |
|---|---|---|---|---|---|---|---|
| WiNGPT2-Llama-3-8B-Base | 继续预训练 (20G) | 8192 | bf16 | 5e-5 | 0.05 | 2 | A100*8 |
| WiNGPT2-Llama-3-8B-Chat | 微调/对齐 (50万条) | 8192 | bf16 | 5e-6 | 0.01 | 4 | A100*8 |
| Type | MCKQuiz | MSceQA | |
|---|---|---|---|
| WiNGPT-Llama-3-8B-Base | Continued Pre-training | 66.3 | / |
| Meta-Llama-3-8B | Pre-training | 37 | / |
| WiNGPT-Llama-3-8B-Chat | Finetuning/Alignment | 65.2 | 79.8 |
| Meta-Llama-3-8B-Instruct | Finetuning/Alignment | 49.8 | 76.3 |
| Meta-Llama-3-70B-Instruct-AWQ | Finetuning/Alignment | 73.5 | 78.6 |