All +## and +### tokens were replaced with new, randomly-selected 2- and 3-digit numbers in the vocab.json and tokenizer.json. You can identify these in outputs because the new tokens start with ^^.
Input and output embeddings for +## and +### tokens were moved to the +00 and +000 embeddings.
Removed associations between numbers from merges.txt
Using a library such as ecco, probabilities for next number token look equally likely, with +000 preferred.
1@article{radford2019language,
2 title={Language Models are Unsupervised Multitask Learners},
3 author={Radford, Alec and Wu, Jeff and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya},
4 year={2019}
5}
