Public multiple-choice benchmarks serialised into the binary task format that
llama-perplexity --multiple-choice reads, so they can be scored directly by
llama.cpp without an evaluation framework in between.
llama-perplexity -m model.gguf -c 512 -np 16 --multiple-choice -bf medqa-test.bin
Pass these with -bf, never -f. They are binary and contain 0x1A bytes,
which a Windows text-mode read treats as end-of-file. With -f llama.cpp prints
a… See the full description on the dataset page:
https://huggingface.co/datasets/nehalahmedshaikh/llama-mc-datasets.
