A fixed evaluation benchmark (1,000 samples) for assessing DPO-trained LLM judges on materials science synthesis recipe evaluation in a reference-free setting.

Purpose

This test set supports two evaluation methods:

1. Reward Accuracy (Log-Probability)

Use prompt, chosen, and rejected to compute implicit reward accuracy without generation: reward_chosen = log P(chosen | prompt) reward_rejected = log P(rejected | prompt) accuracy =… See the full description on the dataset page: https://huggingface.co/datasets/iknow-lab/JudgeBias-DPO-RefFree-testset.