This dataset contains the results of memorization evaluations for all Pythia models. For each model, the dataset lists every training sequence that the fully trained model has memorized.
A training sequence is considered memorized if, when prompted with the first 32 tokens of the sequence, the model's greedy continuation exactly matches the next 32 tokens. This is evaluated over all ~146M training sequences in the Pile.
This dataset was generated for the paper… See the full description on the dataset page:
https://huggingface.co/datasets/EleutherAI/pythia-memorized-evals.
