data set: 76 concept images + around the same number of custmized reg images
learning rate 1.5e-6 or 2e-6 for 5000 steps
text encoder rate 15%
Problems:
It turns out this model has a bunch of problems and is quite hard to use.
On one hand, it feels like the model is undertrained as it often fails to capture the key characteristics of the characters, and in particular often has problem with hairstyles.
On the other hand, it feels like the model is overtrained as we have little flexibility on the style and the background. It tends to throw just red or pink single color background.
Overall, when we try to alter the background or other things, the character gets altered as well. This is clearly due to the bias of the dataset. A more diverse dataset will be needed to improve the model.
.png)
.png)
.png)