Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
llm-course-hw2-reward-model – AI Model by Aleks2002SH | AlphaNeural AI
You can deploy this model and start earning money today!
Aleks2002SH
/
llm-course-hw2-reward-model
like
0
transformers
safetensors
llama
text-classification
generated_from_trainer
trl
reward-trainer
HumanLLMs/Human-Like-DPO-Dataset
HuggingFaceTB/SmolLM-135M-Instruct
finetune
autotrain_compatible
text-generation-inference
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Model Card for llm-course-hw2-ppo-reward-model
Model Description
This model is used to predict reward scores for outputs generated by LLM. It was then used to evaluate and guide training of policy model.
Training Details
the model is a fine-tuned version of pre-trained sequence classification model
SmolLM-135M-Instruct
It was trained on
Human-Like-DPO-Dataset
It was trained to score the generated text based on quality
Parameters of training:
learning rate - 3e-4
epochs - 1
batch size - 16
max length - 1024
The training procedure was done by RewardTrainer from
trl
library
The final loss after 1 epoch - 0.1226
Model Evaluation
chosen answer score for dataset['test'][0]["chosen"] - -0.2461
rejected answer score for dataset['test'][0]["rejected"] - -3.1875
