Views
No views yet
AutoBlocks that folds
text-to-image, img2img, and differential diffusion into a single pipeline, the workflow is chosen automatically
from which inputs you pass.prompt -> text2image
prompt + image -> image2image (optional strength)
prompt + image + diffdiff_map -> differential diffusion1import torch
2from diffusers import ModularPipeline
3
4pipe = ModularPipeline.from_pretrained("OzzyGT/ideogram4_custom_blocks", trust_remote_code=True)
5pipe.load_components(
6 names=["text_encoder", "tokenizer", "transformer", "unconditional_transformer", "vae", "scheduler"],
7 torch_dtype=torch.bfloat16,
8)
9pipe.to("cuda")
10
11image = pipe(prompt="a photo of a red apple on a black background", height=1024, width=1024, output="images")[0]
12image.save("out.png")image (and optional strength) to the same pipe:1from diffusers.utils import load_image
2
3image = load_image("https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/differential/20240329211129_4024911930.png")
4
5result = pipe(
6 prompt="a photo of a snowy mountain landscape at sunset, dramatic clouds",
7 image=image,
8 strength=0.6,
9 output="images",
10)[0]
11result.save("img2img.png")diffdiff_map) alongside the image — darker regions change more, brighter regions stay closer to the reference:1from diffusers.utils import load_image
2
3image = load_image("https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/differential/20240329211129_4024911930.png")
4diffdiff_map = load_image("https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/differential/gradient_mask.png")
5
6result = pipe(
7 prompt="a photo of a snowy mountain landscape at sunset, dramatic clouds",
8 image=image,
9 diffdiff_map=diffdiff_map,
10 output="images",
11)[0]
12result.save("diffdiff.png")