The primary use of JudgeLM is research on evaluating the performance of large language models and chatbots.
The primary intended users of the model are researchers and hobbyists in natural language processing, machine learning, and artificial intelligence.
JudgeLM v1.0 is fine-tuned from Vicuna-v1.3 with supervised instruction fine-tuning.
The training data is around 200K judge samples from JudgeLM-100K dataset.
See more details in the "Fine-tuning Settings" section in the appendix of this paper.
Evaluation
JudgeLM is evaluated on JudgeLM val set, with judgements produced by GPT-4 teacher. See more details in this paper and try it with code.
Additional Information
Citation Information
@article{zhu2023judgelm,
title={JudgeLM: Fine-tuned Large Language Models are Scalable Judges},
author={Lianghui Zhu and Xinggang Wang and Xinlong Wang},
year={2023},
eprint={2310.17631},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
