This is a reward model trained for RLHF alignment, selected from multiple experimental runs based on validation accuracy and reward alignment quality.

Selection Criteria

The best checkpoint was chosen according to:

Highest val_accuracy among all final checkpoints
Minimum reward_alignment_score threshold of 0.80

Only checkpoints satisfying both conditions were eligible.

Training Runs Comparison

Run	Base Model	Learning Rate	Final Step	Val Accuracy	Reward Alignment	Train Loss
run_gpt2_base_lr1e4	GPT-2 Base	1e-4	1000	0.907	0.876	0.115
run_gpt2_base_lr5e5	GPT-2 Base	5e-5	1000	0.870	0.839	0.207
run_gpt2_large_lr1e4	GPT-2 Large	1e-4	1000	0.958	0.928	0.061
run_gpt2_large_lr5e5	GPT-2 Large	5e-5	1000	0.901	0.854	0.159
run_deberta_lr1e4	DeBERTa-v2	1e-4	1000	0.837	0.827	0.301

Best Run Metrics

Metric	Value
Run Name	run_gpt2_large_lr1e4
Val Accuracy	0.958
Reward Alignment Score	0.928
Final Train Loss	0.061

Intended Uses

This model is intended for use as a reward model in RLHF pipelines to score and rank model outputs based on human preference alignment.

How to Use

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained("BestRewardModel-TestRepo")
tokenizer = AutoTokenizer.from_pretrained("BestRewardModel-TestRepo")

inputs = tokenizer("prompt", "response", return_tensors="pt")
score = model(**inputs).logits[0].item()

License

Apache-2.0

dusersad12/BestRewardModel-TestRepo

BestRewardModel

Model Description