This model is a fine-tuned reward model based on HuggingFaceTB/SmolLM-135M-Instruct, trained on the HumanLLMs/Human-Like-DPO-Dataset dataset.
It has been trained using TRL to evaluate and rank responses based on human preferences, playing a crucial role in RLHF (Reinforcement Learning from Human Feedback) for models like SmolLM-135M-PPO.

📝 Overview

Base Model: SmolLM-135M-Instruct
Fine-Tuned Dataset: HumanLLMs/Human-Like-DPO-Dataset
Objective: Learn to assign higher scores to more engaging, structured, and emotional responses.
Use Case: Used in PPO-based RLHF training to reinforce human-like response quality.

Training Method

The model was fine-tuned using Direct Preference Comparisons:
- Each sample contains a chosen response (preferred) and a rejected response.
- The model learns to assign higher rewards to the chosen response and lower rewards to the rejected one.
This reward function was used in PPO fine-tuning to optimize response generation.

tsessk/llm-course-hw2-reward-model

🏆 Model Card for llm-course-hw2-reward-model

📝 Overview

Training Method

tsessk/
llm-course-hw2-reward-model