This model was trained for the purposes of analysing model utility when trained on various
Derived Text Formats.
These are versions of the same text that are adjusted to reduce the chances that the original text can ever be extracted from the model, with applications in privacy and copyright infringement protection.
In this case, the model was trained on the dataset after turning each text into a Bag-of-Words format (i.e. scrambling the words without concern for hierarchical structure)
The dataset used for these experiments is
codelion/fineweb-edu-1B, with all obfuscated formats found
here.
