You can deploy this model and start earning money today!

mario-rc/
multi-domain-rm-qwen-3-nemotron-8b-it

Name: multi-domain-rm-qwen-3-nemotron-8b-it
Availability: InStock

Views

No views yet

Multi-Domain Reward Model Qwen3-Nemotron-8B-Instruct

This is a multi-domain reward model built from nvidia/Qwen3-Nemotron-8B-BRRM. It combines 23 fine-grained regression objectives across coherence, commonsense, empathy, and multicultural response quality with a prompt-conditioned gating network that produces a single preference score.

The checkpoint was packaged with the custom RewardModelWithGating architecture used in the Multi-Domain Reward Model project. Its shared-prompt gate is computed once and reused for both responses in each preference pair.

Project repository: Mario-RC/multi-domain-reward-model.

Intended use

Use this model to score and compare assistant responses when the evaluation should account for multiple quality dimensions rather than a single generic helpfulness score. The primary use cases are reward modeling, preference ranking, reranking, and offline alignment evaluation for chat-style data.

Training data

The model was trained with data from the multidomain_data_scoring project:

Multi-Domain-Data-Scoring
Multi-Domain-Data-Preference-Pairs-SharedGate

Evaluation

Results on the held-out multi-domain test set:

Metric	Result
Test accuracy (%)	83.65
Scoring Spearman	0.6704
Coherence accuracy	72.81%
Commonsense accuracy	96.21%
Empathy accuracy	89.99%
Multicultural accuracy	67.24%

Hugging Face Models

Model	Base reward model	Test accuracy (%)	Scoring Spearman
`multi-domain-rm-fsfairx-gemma-2-9b-it`	sfairXC/FsfairX-Gemma2-RM-v0.1	88.01	0.7346
`multi-domain-rm-skywork-qwen-3-8b-it`	Skywork/Skywork-Reward-V2-Qwen3-8B	87.82	0.7156
`multi-domain-rm-fsfairx-llama-3-8b-it`	sfairXC/FsfairX-LLaMA3-RM-v0.1	86.86	0.7108
`multi-domain-rm-skywork-llama-3.1-8b-it`	Skywork/Skywork-Reward-V2-Llama-3.1-8B	86.82	0.7264
`multi-domain-rm-mistral-7b-it`	weqweasdas/RM-Mistral-7B	84.41	0.6710
`multi-domain-rm-qwen-3-nemotron-8b-it`	nvidia/Qwen3-Nemotron-8B-BRRM	83.65	0.6704

Usage

The repository includes custom Transformers code, so trust_remote_code=True is required. Compute the gate once from the prompt and reuse that tensor when scoring both complete candidates.

python

import torch
from transformers import AutoModel, AutoTokenizer

repo_id = "mario-rc/multi-domain-rm-qwen-3-nemotron-8b-it"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModel.from_pretrained(
    repo_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
).eval()

prompt = [{"role": "user", "content": "How can I support a friend who feels excluded?"}]
chosen = prompt + [{
    "role": "assistant",
    "content": "Listen without judging, validate how they feel, and ask what support would help.",
}]
rejected = prompt + [{"role": "assistant", "content": "Tell them to ignore it."}]

prompt_ids = tokenizer.apply_chat_template(
    prompt,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)
chosen_ids = tokenizer.apply_chat_template(
    chosen,
    tokenize=True,
    add_generation_prompt=False,
    return_tensors="pt",
).to(model.device)
rejected_ids = tokenizer.apply_chat_template(
    rejected,
    tokenize=True,
    add_generation_prompt=False,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    gate = model.compute_gating(input_ids=prompt_ids)
    chosen_score = model(
        input_ids=chosen_ids,
        gating_output_override=gate,
    ).score
    rejected_score = model(
        input_ids=rejected_ids,
        gating_output_override=gate,
    ).score

print({"chosen": chosen_score.item(), "rejected": rejected_score.item()})

For padded batches, also pass the matching attention_mask. Scores are intended for comparison within a prompt; they are not calibrated probabilities or universal utility values.

Limitations

This is a reward model, not a standalone chat assistant. Scores are intended for relative comparison and should be calibrated for each downstream use case. Performance can vary by language, topic, and distribution. The model inherits limitations and biases from its base model and training data and should not be used as the sole decision-maker in high-impact settings.

Credits

This model is based on the ArmoRM/RLHFlow reward-modeling approach and adapts it to custom multi-domain attributes for coherence, commonsense, empathy, and multicultural response quality.

License

The project code is released under Apache-2.0. Use of this checkpoint is also subject to the license and usage conditions of the base model and training datasets.