Views
No views yet
transformers library.
To use the fine-tuned model for text generation based on a persona, follow these steps:1from transformers import pipeline, AutoModelForCausalLM, AutoTokenizer
2
3# Load the fine-tuned model and tokenizer
4model_name = "hello12w/persona_chatbot"
5model = AutoModelForCausalLM.from_pretrained(model_name)
6tokenizer = AutoTokenizer.from_pretrained(model_name)
7
8# Define the persona and prompt
9prompt = prompt = f"""
10
11Person B has the following Persona information.
12
13Persona of Person B: My name is Sarah and I'm a 28 year old software engineer.
14
15Persona of Person B: I love coding and developing new software applications.
16
17Persona of Person B: In my free time, I enjoy reading sci-fi novels and playing board games.
18
19Instruct: Person A and Person B are now having a conversation.
20
21Following the conversation below, write a response that Person B would say based on the above Persona information.
22
23Please carefully consider the flow and context of the conversation below, and use the Person B's Persona information appropriately to generate a response that you think is the most appropriate reply for Person B.
24
25Persona A: Hi Sarah, I heard you're working on a cool project at work. Can you tell me more about it?
26
27Output:
28
29"""
30input_ids = tokenizer(prompt, return_tensors="pt", truncation=True)
31attention_mask = input_ids.attention_mask
32input_ids = input_ids.input_ids
33
34# Inference
35with torch.no_grad():
36 outputs = model.generate(
37 input_ids=input_ids,
38 attention_mask=attention_mask,
39 max_new_tokens=200,
40 do_sample=True,
41 top_p=0.95,
42 temperature=0.9
43 )
44
45# Decode output tokens
46decoded_outputs = tokenizer.batch_decode(outputs, skip_special_tokens=True)
47output = decoded_outputs[0][len(prompt):]
48
49print(output)