8/1 - Added Hermes 3 Dataset
A mixture of synthetic data pulled from all over HuggingFace and then aggressively deduplicated. Nearly 20gb of synth data crunched down to just under 2Gb.
Also cleaned up any system prompts which would not be expected to alter behavior.
Mostly done as an excercise in deduplication and ngram analysis. Used to finetune
https://huggingface.co/cgato/Nemo12b-TheSyntheticOne
If you use this dataset you need to mask out the samples labeled false, not doing so will… See the full description on the dataset page:
https://huggingface.co/datasets/cgato/TheSmarts.
