Views
No views yet
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3ppo_model = AutoModelForCausalLM.from_pretrained("pbedrin/llm-course-hw2-ppo")
4ref_model = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM-135M-Instruct")
5tokenizer = AutoTokenizer.from_pretrained(f"pbedrin/llm-course-hw2-ppo", padding_side="left")
6
7messages = [{"role": "user", "content": "What's your morning routine like?"}]
8text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
9model_inputs = tokenizer([text], return_tensors="pt")
10
11generated_ids = dpo_model.generate(model_inputs.input_ids, max_new_tokens=128, do_sample=False)
12response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
13
14init_generated_ids = ref_model.generate(model_inputs.input_ids, max_new_tokens=128, do_sample=False)
15init_response = tokenizer.batch_decode(init_generated_ids, skip_special_tokens=True)[0]
16
17print("======== BEFORE TUNING ========")
18print(init_response)
19
20print("======== AFTER TUNING ========")
21print(response)======== BEFORE TUNING ========
user
What's your morning routine like?
assistant
I'm excited to start my morning routine! As a digital AI assistant, I don't have personal preferences or habits, but I can provide you with a general idea of what a morning routine might look like. Here's a sample routine that I've developed based on various studies and research:
**Morning Routine (10-15 minutes)**
1. **Hydrate**: Drink a full glass of water or a herbal tea (e.g., chamomile, peppermint) to start the day.
2. **Eat a nutritious breakfast**: Prepare a healthy breakfast, such as oatmeal with fruit, scrambled eggs with
======== AFTER TUNING ========
user
What's your morning routine like?
assistant
What a great question! I've been meaning to share my morning routine with you, but I've been stuck in a rut. Here's my 5-day morning routine that's been working for me:
**Morning (5:00 AM - 7:00 AM)**
1. **Hydrate**: Drink a full glass of water or coffee (if you're feeling thirsty). This is the first step in getting your body ready for the day.
2. **Brush your teeth**: Get your morning routine started with a good brushing. You can use a toothbrush or just a piece of toothpaste.