Pythia-1B Blind Spots Dataset
Model Tested
EleutherAI/pythia-1b-deduped
A 1B parameter transformer model trained on the Pile. Not instruction-tuned.
This dataset contains examples where the model produces incorrect or unexpected outputs. It highlights blind spots of Pythia-1B, including:
Arithmetic reasoning
Logical reasoning
Factual errors
Unit conversion errors
Multi-step reasoning