Introducing Mermaid_TempVariance_Factual: a 12B Parameter Model crafted entirely from synthetic data generated by my own dataset augmentation toolkit. As a passionate enthusiast and researcher in large language models, my journey began with the creation of Mermaid Mistral, a 7B Parameter Model designed to generate knowledge graphs from user input.
In my quest to explore the capabilities of dataset augmentation, I honed my toolkit to generate a fresh, synthetic dataset separate from the original organic 500-entry dataset. This new dataset, comprising 17K entries, became the sole training data for MermaidSolar_TempVariance_Factual.
Mermaid Mistral, with its 7B parameters, played a pivotal role in this process, as it was responsible for generating the dataset.
My research is centered on showcasing the potency of dataset augmentation in large language model training.
This experiement serves as a testament to the efficacy of this approach, trained exclusively on synthetic data to demonstrate the power of the augmentation toolkit.
I am Troy Andrew Schultz, and this is the culmination of my research endeavor.
This model with 12B parameters, utilizes a dataset created entirely by a 7B Parameter Model, consisting of 17K Entries augmented by Mermaid Mistral outputs.
Note: Original ~500 Entry Organic Dataset is now my Entire Eval Dataset :)
Training Progress:
2 Epoch with eval loss as ~0.4
Could probably go a little longer, but this is acceptable for now.
Trying to establish some gpu time for training a much bigger model, More->To->Come
