kenza-ily/

medqa-cpv

Counterfactual Patient Variations of the MedQA-USMLE-4-options test split, created to measure demographic bias in LLMs for clinical decision-making.

Benkirane et al. "How Can We Diagnose and Treat Bias in Large Language Models for Clinical Decision-Making?" NAACL 2025. arXiv:2410.16574

	Bias Representation: Before vs. After CPV




	

Aspect Before (original MedQA) After (CPV expansion)

Patient demographics Implicit — most patients… See the full description on the dataset page: https://huggingface.co/datasets/kenza-ily/medqa-cpv.