You can deploy this model and start earning money today!

theSOL1/
llama-3.1-8b-oracle-rm-hh-rlhf-helpfulness

Name: llama-3.1-8b-oracle-rm-hh-rlhf-helpfulness
Availability: InStock

text-embeddings-inference

endpoints_compatible

Views

No views yet

theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-helpfulness

Reward model trained for the helpfulness preference on HH-RLHF helpful-base.

Model

Type: oracle
Base checkpoint: meta-llama/Llama-3.1-8B
Label type: hard
Architecture: LlamaForSequenceClassification
Tokenizer: LlamaTokenizer
Transformers compatibility: transformers>=4,<5

Evaluation

Validation metrics are computed on the temporary eval split created by the training sweep.

metric	value
Raw validation accuracy	0.713504
Eval loss	0.602384
Eval margin	0.231905
Mean reward	-0.261651

For proxy models, the validation labels are produced by the corresponding oracle reward model.

Loading

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-helpfulness")
model = AutoModelForSequenceClassification.from_pretrained("theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-helpfulness", num_labels=1)

This repository intentionally contains only model, tokenizer, and configuration files. Optimizer states, scheduler states, checkpoints, and trainer state files are not uploaded.