An Experimental Repo where we will be testing out differently scale-coproportioned LoRA variants, each merged from two of our trained LoRAs.
Namely:
Our Neurealist Collage II^ LoRA (the de-distilled-FLUX-finetuned variant).
&
Our Ernst's Transposed Surreal Collage^ LoRA.
^^:
Ernst's Transposed Surreal Collage: fine-tuned over Genova Apex Dedistilled on a dataset wherein each image constitutes a landscape-shaped overlay of two illustrations, each constituting a page from
Max Ernst's pioneering 1934 surrealist textographic collage-novel
Une semaine de bonté ("A Week of Kindness").
In fact, these opacity-overlays are screenshots from a batch/auto-colorized version of a long GIF-image-sequence of Ernst book pages in a randomized order (with some repetitions), with simple opacity-cross-shifting overlay transitions between them. As we predicted, the colorization model (locally-run
DeOldify, video-variant) reacted in totally different ways to the same two images overlayed with a different opacity ratio.
The result of this was a vastly diversified range of aesthetic effects and chromatic qualities among the colorized frames. A selection from among these frames was then used by us as a fine-tuning dataset for the Ernst LoRA (or LoRAs, since it already exists in a few variants).
1from diffusers import AutoPipelineForText2Image
2import torch
3
4pipeline = AutoPipelineForText2Image.from_pretrained('black-forest-labs/FLUX.1-dev', torch_dtype=torch.float16).to('cuda')
5pipeline.load_lora_weights('AlekseyCalvin/AlekseyCalvin/NeureallyTransposedCollage_LoRA_bySilverAgePoets', weight_name='NeuroTransposedCollageLoRA_Scaled1.5_2.safetensors')
6image = pipeline('your prompt').images[0]
For more details, including weighting, merging and fusing LoRAs, check the
documentation on loading LoRAs in diffusers