Addition problems in the format {a} + {b} = {c}.
test: 5K held-out evaluation examples (operands >= 10, i.e. min 2 digits)
1BT: 85M training examples (1 billion tokens under Llama-3 tokenizer)
10BT: 850M training examples (10 billion tokens)
3MT-3digit: Exhaustive single-token addition: all (a, b) with a, b in [0, 999] and a+b <= 999. 500,500 ordered pairs, ~3M tokens. All of a, b, c are single tokens. Symmetry-safe train/test split (10% test).… See the full description on the dataset page:
https://huggingface.co/datasets/deqing/addition_dataset.
