Download the HuggingFaceH4/MATH-500 dataset and sample 20 problems at random from levels 1 to 3.
Run inference with temperature T=1 and N=16 samples using the model Qwen/Qwen2.5-1.5B-Instruct and CoT system prompt asking the model to put the answer in \boxed.
Use majority voting to decide an answer from these samples.
The column question has the sampled questions, all_responses include model's… See the full description on the dataset page:
https://huggingface.co/datasets/mertunsal/HFAssessment_eval_16.
