Views
No views yet
[!IMPORTANT] NOTE: This feature is not officially supported yet. You'll need to install the repo from this pull request.
1from OmniGen import OmniGenPipeline, OmniGen
2
3# pass the quantized model in the pipeline
4model = OmniGen.from_pretrained('gryan/OmniGen-v1-bnb-4bit')
5pipe = OmniGenPipeline.from_pretrained("Shitao/OmniGen-v1", model=model)
6
7# proceed as normal!
8
9## Text to Image
10images = pipe(
11 prompt="A curly-haired man in a red shirt is drinking tea.",
12 height=1024,
13 width=1024,
14 guidance_scale=2.5,
15 seed=0,
16)
17images[0].save("example_t2i.png") # save output PIL Image
18
19## Multi-modal to Image
20# In the prompt, we use the placeholder to represent the image. The image placeholder should be in the format of <img><|image_*|></img>
21# You can add multiple images in the input_images. Please ensure that each image has its placeholder. For example, for the list input_images [img1_path, img2_path], the prompt needs to have two placeholders: <img><|image_1|></img>, <img><|image_2|></img>.
22images = pipe(
23 prompt="A man in a black shirt is reading a book. The man is the right man in <img><|image_1|></img>.",
24 input_images=["./imgs/test_cases/two_man.jpg"],
25 height=1024,
26 width=1024,
27 guidance_scale=2.5,
28 img_guidance_scale=1.6,
29 seed=0
30)
31images[0].save("example_ti2i.png") # save output PIL image
| Settings | Only Text | Text + Single Image | Text + Two Images |
|---|---|---|---|
| use_kv_cache=False | 6.8G, 1m16s | 7.2G, 3m30s | 7.7G, 5m47s |
| use_kv_cache | 9.9G, 1m14s | 20.4G†, 8m5s | OOM (36.7G†, >1h10m) |
| use_kv_cache,offload_kv_cache | 6.8G, 1m16s | 7.2G, 2m49s | 8.4G, 4m3s |
| use_kv_cache,offload_kv_cache,separate_cfg_infer | 6.8G, 1m20s | 7.0G, 2m31s | 7.4G, 3m31s |
| use_kv_cache,offload_kv_cache,offload_model* | 5.0G, 1m35s | 6.0G, 3m7s | 8.0G, 4m21s |
| use_kv_cache,offload_kv_cache,separate_cfg_infer,offload_model* | 5.0G, 1m58s | 5.3G, 3m29s | 5.6G, 4m19s |