Views
No views yet
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_path = "WiNGPT2-Gemma-2-9B-Chat"
4device = "cuda"
5
6tokenizer = AutoTokenizer.from_pretrained(model_path)
7model = AutoModelForCausalLM.from_pretrained(model_path).to(device)
8model = model.eval()
9
10
11messages = [{"role": "user", "content": "WiNGPT, 你好"}]
12input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, return_tensors='pt')
13
14output_ids = model.generate(
15 input_ids.to(device),
16 eos_token_id=tokenizer.convert_tokens_to_ids('<end_of_turn>'),
17 max_new_tokens=1024,
18 repetition_penalty=1.1
19 )
20response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)
21
22print(response)
23
24## 输出结果示例：你好！今天我能为你做些什么？{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<start_of_turn>' + message['role'] + '\n' + message['content'] + '<end_of_turn>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<start_of_turn>assistant\n' }}{% endif %}[{"role": "user", "content": "WiNGPT, 你好"}]1[
2 {"role": "user", "content": "WiNGPT, 你好"},
3 {"role": "assistant", "content": "你好！今天我能为你做些什么？"},
4 {"role": "user", "content": "流感应该怎么办啊？"},
5]1[
2 {"role": "system", "content": "作为医疗领域的智能助手，WiNGPT将提供中英翻译服务。用户输入的中文或英文内容将由WiNGPT进行准确的翻译，以满足用户的语言需求。"},
3 {"role": "user", "content": "Life is short, you know, and time is so swift; Rivers are wide, so wide, and ships sail far."}
4]| 名称 | 训练策略 | 长度 | 精度 | 学习率 | Weight_decay | Epochs | GPUs |
|---|---|---|---|---|---|---|---|
| WiNGPT2-Gemma-2-9B-Base | 继续预训练 (23G) | 8192 | bf16 | 5e-5 | 0.05 | 2 | A100*8 |
| WiNGPT2-Gemma-2-9B-Chat | 微调/对齐 (45万条) | 8192 | bf16 | 5e-6 | 0.012 | 2 | A100*8 |
| Type | MCKQuiz-2.0 / (2024only) | MSceQA-2.0 | |
|---|---|---|---|
| WiNGPT2-Gemma-2-9B-Base | Continued Pre-training | 74.3 / 77.8 | / |
| WiNGPT2-Llama-3-8B-Base | Continued Pre-training | 66.3 / 70.4 | / |
| gemma-2-9b | Pre-training | 44.8 / 42.0 | / |
| WiNGPT2-Gemma-2-9B-Chat | Finetuning/Alignment | 73.5 / 77.8 | 82.9 |
| gemma-2-9b-it | Finetuning/Alignment | 53.7 / 48.2 | 80.18 |
| Llama-3.1-8B-Instruct | Finetuning/Alignment | 61.6 / 68.5 | 73.2 |
| WiNGPT2-Llama-3-8B-Chat | Finetuning/Alignment | 65.8 / 72.2 | 73.0 |