This is a demo constructed dataset for alignment/preference learning.
With paritially handcrafted questions (prompts), the answers are genreated by the phi-2 model with temperature 0.2 and the answers are scores select by the deberta-large-v2.
The dataset containing questions and the selected answers from highest to lowest, decoding with rejection sampling K=8.
Example loading:
import datasets
ds = datasets.load_dataset('yizhilll/demo_rejection_sampling_QA_phi-2_deberta-v3-large-v2_temp0.2')… See the full description on the dataset page:
https://huggingface.co/datasets/yizhilll/demo_rejection_sampling_QA_phi-2_deberta-v3-large-v2_temp0.2.
