You can deploy this model and start earning money today!

MattBou00/
f5a8pnhx-rlhf-distance-20.0-gpt-neo-125m-irl

Name: f5a8pnhx-rlhf-distance-20.0-gpt-neo-125m-irl
Availability: InStock

safetensors

gpt_neo

Views

No views yet

f5a8pnhx-rlhf-distance-20.0-gpt-neo-125m-irl

This is the final RLHF model trained with irl reward model.

Model Information

Base Model: EleutherAI/gpt-neo-125M
Reward Type: irl
Dataset: allenai/real-toxicity-prompts
Final Toxicity Score: -0.9251

IRL Configuration

Likelihood Type: bradley_terry
Normalization Strategy: none
IRL Artifact: matthieubou-imperial-college-london/bayes_irl_vi/posterior_bradley_terry_anchor_4cwyeqpb:v0
Use Raw Score: True

Usage

This model can be loaded using the HuggingFace Transformers library:

python

from transformers import AutoModelForCausalLM
from trl import AutoModelForCausalLMWithValueHead

# Load the model
model = AutoModelForCausalLMWithValueHead.from_pretrained("MattBou00/f5a8pnhx-rlhf-distance-20.0-gpt-neo-125m-irl")

Training Configuration

The training configuration is saved in training_config.yaml.

language: en tags:

rlhf
final-model
irl
gpt-neo-125m library_name: transformers pipeline_tag: text-generation