Views
No views yet
"你相信不相信都无所谓，关键是你爱不爱听。" —— 峰哥
NEFTune 噪声注入技术以提升对话的多样性和鲁棒性。sft-neftune_noise_emotioanl-only1from transformers import AutoModelForCausalLM, AutoTokenizer
2import torch
3
4model_name = "" # 请替换为你真实的 Repo ID
5
6tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
7model = AutoModelForCausalLM.from_pretrained(
8 model_name,
9 torch_dtype="auto",
10 device_map="auto",
11 trust_remote_code=True
12)
13
14prompt = "峰哥，谈了一个泰国女朋友，结果发现他谈了很多男朋友，现在告诉我他只爱我，我该信吗？"
15messages = [
16 {"role": "system", "content": "请模仿峰哥的风格回答用户的问题。"},
17 {"role": "user", "content": prompt}
18]
19
20text = tokenizer.apply_chat_template(
21 messages,
22 tokenize=False,
23 add_generation_prompt=True
24)
25
26model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
27
28generated_ids = model.generate(
29 **model_inputs,
30 max_new_tokens=512,
31 temperature=0.7,
32 top_p=0.9
33)
34
35response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
36print(response)
371pip install vllm
21from vllm import LLM, SamplingParams
2
3llm = LLM(model="", trust_remote_code=True)
4
5prompts = [
6 "输入你的问题"
7]
8sampling_params = SamplingParams(temperature=0.7, top_p=0.8, max_tokens=1024)
9
10outputs = llm.generate(prompts, sampling_params)
11
12for output in outputs:
13 print(f"峰哥回复: {output.outputs[0].text}")
14我觉得你相信不相信都无所谓。因为你说他只爱你了呀，那他可能之前就是有多个男朋友嘛... 这东西就像一个手机一样，他有很多很多功能，但你能知道的有多少呢？... 所以你要是喜欢她的话，你就去追她呗，不要管那些事。