You can deploy this model and start earning money today!

koala1230/
Qwen3-8B-ORM-MixedPreference

Name: Qwen3-8B-ORM-MixedPreference
Availability: InStock

safetensors

qwen3

Skywork/Skywork-Reward-Preference-80K-v0.2

nvidia/HelpSteer3

Qwen/Qwen3-8B

finetune

Views

No views yet

This repository contains an Outcome Reward Model (ORM) fine-tuned from Qwen/Qwen3-8B, trained on the pairwise preference dataset Skywork/Skywork-Reward-Preference-80K-v0.2 and nvidia/Helpsteer3.

Key Details

Base Model: Qwen/Qwen3-8B
Training Dataset: Skywork/Skywork-Reward-Preference-80K-v0.2 and nvidia/Helpsteer3
Loss Function: Bradley-Terry loss

Training Configuration:
Epochs: 1
Optimizer: 2e-6 learning rate
Scheduler: Cosine decay with warmup
Batch Size: 32

Usage

You can load the model using the HuggingFace transformers library:

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer

# load model and tokenizer
rm = AutoModelForSequenceClassification.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    num_labels=1,
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

Evaluation

Performance on RewardBench 2

Model	Score	Factuality	Precise IF	Math	Safety	Focus	Ties
koala1230/Qwen3-8B-ORM-MixedPreference	71.4	61.1	49.4	73.2	86.4	92.9	65.5

Performance on RM-Bench

Model	Chat	Math	Code	Safety	Easy	Normal	Hard	Average
koala1230/Qwen3-8B-ORM-MixedPreference	66.5	62.3	59.2	93.1	90.1	75.3	45.4	70.3

License

This model inherits the license of the base model (Qwen3-8B) unless otherwise specified. Please review the original license before use.

koala1230/Qwen3-8B-ORM-MixedPreference

Key Details

Usage

Evaluation

License

koala1230/
Qwen3-8B-ORM-MixedPreference