Domain-level metacognitive monitoring quality in 33 frontier LLMs.
47,151 (answer, confidence) observations from 33 frontier LLMs on 1,500 stratified MMLU items across six cognitive domains.
The Metacognitive Profile Atlas provides item-level verbalized-confidence data for evaluating how well LLMs monitor their own accuracy, decomposed by cognitive domain. Each observation is one (model, item) pair containing the model's… See the full description on the dataset page:
https://huggingface.co/datasets/synthiumjp/metacognitive-profile-atlas.
