Using Stable Diffusion XL 1.5, trained a custom model for image generation of Home Decor usecase
Also applied Canny ControlNets to it for better performance.
Below are the generated images from this model.
These are reemas-irasna/home-decor_LoRA LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0.
The weights were trained using
DreamBooth.
LoRA for the text encoder was enabled: False.
Special VAE used for training: madebyollin/sdxl-vae-fp16-fix.
You should use "a photo of home" to trigger the image generation.
Weights for this model are available in Safetensors format.
Download them in the Files & versions tab.
1import torch
2from diffusers import DiffusionPipeline, AutoencoderKL
3
4vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
5pipe = DiffusionPipeline.from_pretrained(
6 "stabilityai/stable-diffusion-xl-base-1.0",
7 vae=vae,
8 torch_dtype=torch.float16,
9 variant="fp16",
10 use_safetensors=True
11)
12pipe.load_lora_weights("reemas-irasna/home-decor_LoRA")
13_ = pipe.to("cuda")
14
15prompt = "a photo of bedroom in red and white combination"
16
17image = pipe(prompt=prompt, num_inference_steps=25).images[0]
18image