This is a highly experimental model in the bigASP family.
Drag this image into ComfyUI to load the example workflow. I've done limited testing so far on running the model, so take that workflow and the following advice with a grain of salt.
-
NOTE: V2.5 is not a standard SDXL model! It's a Flow Matching SDXL! In the example workflow, the ModelSamplingSD3 node is very important to get Comfy to use the model correctly. As for other software beyond Comfy, I don't yet know if they can be coaxed in a similar fashion.
-
First, CFG. You can use the model with or without Perturbed Attention Guidance (PAG). The example workflow includes it. Generally it should be set around 2.0. When using it, you should decrease your CFG lower than usual. Settings between 2 and 5 seem to work okay when PAG is enabled.
If you bypass PAG (connect the ModelSampling node directly to KSampler), then I've generally found CFG should be between 4 and 6. As usual with CFG you're trading off quality for generation diversity. Anything above 7, at least on photorealistic gens, starts to cook the image too much.
-
For negative prompt, I've tried leaving it blank, "low quality", and a more universal long negative prompt with things like "deformed", etc. Of those three options, just "low quality" performed the best for me.
-
For the sampler, euler is the only one I have found that works reliably.
-
For the schedule, keeping the ModelSampling shift parameter at 1.0 and setting the schedule to "beta" works well and provides a good balance between the quality of the structure of the image and the details. Alternatively, you can use the normal schedule and set shift to between 3.0 and 12.0. I've found 6.0 to work best overall, with little benefit from going higher. This configuration will have better overall structure in the image, but details suffer because the model spends more time on the low frequencies. So it depends on what you're trying to generate, preferences, etc. PAG helps with image structure as well, so enabling PAG and using a beta schedule with shift=1.0 is a nice all around good option.
-
As for prompting, I don't yet know what works best. The model was trained on captions generated by JoyCaption Beta One in various modes, so it should understand a variety of ways of prompting. It was also trained on tag strings. I would lean toward well written, natural language captions for now. If you're prompting for NSFW things, try to use more vanilla/clincal terms like those used by chatbots, rather than slang.
-
Images in the training dataset were assessed for overall quality and their quality rating was included during training. So you can prompt with: worst quality, low quality, normal quality, high quality, best quality, and masterpiece quality. Including the quality in your prompt is not required, and it can be included anywhere in the prompt. I would caution against using masterpiece quality for now, since that tends to skew the model towards drawings rather than photoreal.
bigASP V2.5 is an intermediary experiment I wanted to run while I prepared for V3. Like V2, it is a large scale finetune from SDXL, and follows most of V2's approach except for these bizarre tweaks:
Overall it's a very silly experiment, but I wanted to use it as a way to get my feet wet for V3. The upshot is that I'm now far more familiar with the dynamics of flow matching, training at the 100M+ scale, and my training scripts now support multi-node training. I trained V2.5 on four nodes, each with 8xH100sxm5 GPUs, for a total of 32 GPUs. That significantly speeds up the rate at which I can train models.
At this point I might do another blast of training on top of V2.5 with these tweaks:
I suspect I can get a further boost in prompt adherence and generation quality by training the text encoders. What I've noticed between training V2.5 with the encoders frozen, versus V2 with one of the encoders unfrozen, is that the model gets a quick initial boost early in training when it can train the text encoders. My hypothesis is that the Unet is dumping some of the workload into the text encoder. While that might provide a boost to image quality, it might also hurt the model's ability to be flexible and generalize. I think doing a quick 10M additional run, and unfreezing the first couple of layers on the encoders, will strike a nice balance.
And given the odd improvements caused by including "low quality" in negative, I think explictly adding garbage and mangled images to the dataset might actually help the model.
Want to support more dumbass experiments like this, bigASP v3, and JoyCaption?
https://ko-fi.com/fpgaminer
