😊 Use casual/emotional language (+1.2 reward)
💬 Include follow-up questions (+0.8)
❌ Penalizes:
- Overly formal responses (-1.5)
- Generic disclaimers (-2.1)
- Off-topic digressions (-1.8)

Usage

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained("bikmish/llm-course-hw2-reward-model")
tokenizer = AutoTokenizer.from_pretrained("bikmish/llm-course-hw2-reward-model")

text_pair = [
    "That's awesome! Tell me more details! 😄",  # chosen
    "I cannot provide opinions about entertainment."  # rejected
]

inputs = tokenizer(text_pair, padding=True, return_tensors="pt")
rewards = model(**inputs).logits
print(f"Chosen: {rewards[0].item():.2f}, Rejected: {rewards[1].item():.2f}")