I learned AFTER putting all my time into this thing, that I was "doing it wrong"

I should have been training in bf16 "mixed precision", but still saving results in fp32 format. So this is significantly lossy, compared to where it should/could have been. Just leaving this here for historical purposes, but you probably shouldnt use it.

Work in progress... unlike the other uploaded models, this is a bf16 precision model. IF someone wants to mess around with a bf16 version of my xlsd project, here it is.

I am continually refining my datasets. For phase 2, this uses a subset of https://huggingface.co/datasets/opendiffusionai/laion2b-en-aesthetic-square with the original LAION captions. However, I am currently working on recaptioning with InternLM 7b, and then filtering some more. I will then redo training of this "phase 2" step with the updated dataset.

Long term ideal plans would be, once I have optimally cleaned datasets, to redo from scratch, with phase one using the 200k laion square subset, then phase 2 using a square subset of my CC12M https://huggingface.co/datasets/opendiffusionai/cc12m-xlsd (and possibly adding back 1mp sized images) (or possibly switching the order. Not sure yet)

Finally, doing a cleanup with https://huggingface.co/datasets/opendiffusionai/pexels-janpf-sharp