Model Card for Model ID

rm = trainer.model
rm.eval()

def render_pair_for_scoring(source_text, revision_text):
    messages = [
        {
            "role": "user",
            "content": f"{USER_PREFIX}\n\n{str(source_text).strip()}",
        },
        {
            "role": "assistant",
            "content": f"{ASSISTANT_PREFIX}\n\n{str(revision_text).strip()}",
        },
    ]
    return apply_qwen_template_clean(messages)

def reward_score(source_text, revision_text):
    text = render_pair_for_scoring(source_text, revision_text)
    inputs = tokenizer(
        text,
        return_tensors="pt",
        truncation=True,
        max_length=MAX_LENGTH,
    ).to(rm.device)

    with torch.no_grad():
        logits = rm(**inputs).logits

    return logits.squeeze().float().item()

row = split["test"][0]

print("chosen score:  ", reward_score(row["prompt"], row["chosen"]))
print("rejected score:", reward_score(row["prompt"], row["rejected"]))

Model Details

Model Description

• Developed by: [More Information Needed]
• Funded by [optional]: [More Information Needed]
• Shared by [optional]: [More Information Needed]
• Model type: [More Information Needed]
• Language(s) (NLP): [More Information Needed]
• License: [More Information Needed]
• Finetuned from model [optional]: [More Information Needed]

Model Sources [optional]

• Repository: [More Information Needed]
• Paper [optional]: [More Information Needed]
• Demo [optional]: [More Information Needed]

Uses

Direct Use

Downstream Use [optional]

Out-of-Scope Use

Bias, Risks, and Limitations

Recommendations

How to Get Started with the Model

Training Details

Training Data

Training Procedure

Preprocessing [optional]

Training Hyperparameters

• Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

Evaluation

Testing Data, Factors & Metrics

Testing Data

Factors

Metrics

Results

Summary

Model Examination [optional]

Environmental Impact

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

Compute Infrastructure

Hardware

Software

Citation [optional]

Glossary [optional]

More Information [optional]

Model Card Authors [optional]

Model Card Contact