For the #huggan event we decided to fine-tune a cloob-conditioned latent diffusion model.
Training largely followed the guidelines in
https://github.com/JD-P/cloob-latent-diffusion
The (lightly modified) training script is included in this model repo, but I'd recommend starting in the github repository linked above.
The model was fine-tuned on Wikiart (huggan/WikiArt) for ~12 hours on 2 A6000 GPUs kindly provided by Paperspace.
The large files in this repository are
The final component is CLOOB, but the code for sampling will include downloading the right pretrained CLOOB model since we didn't modify that bit.
More info coming soon.
For now if you really want to try it, this colab (
https://colab.research.google.com/drive/1HPu6tz44brMnKOU4G6Luhy1_iMgDl1fw#scrollTo=2s0I4RksiHtA) will download the models and set up a gradio interface for it.
