This pipeline was finetuned from kandinsky-community/kandinsky-2-2-decoder on the waitwhoami/vqa_caption.dataset-full dataset. Below are some example images generated with the finetuned pipeline using the following prompts: ['The colonoscopy image contains a single, moderate-sized polyp that has not been removed, appearing in red and pink tones in the center and lower areas.']:

Pipeline usage

You can use the pipeline like so:

python

from diffusers import DiffusionPipeline
import torch

pipeline = AutoPipelineForText2Image.from_pretrained("SushantGautam/kandi2-decoder-medical-model", torch_dtype=torch.float16)
prompt = "The colonoscopy image contains a single, moderate-sized polyp that has not been removed, appearing in red and pink tones in the center and lower areas."
image = pipeline(prompt).images[0]
image.save("my_image.png")

Training info

These are the key hyperparameters used during training:

Epochs: 30
Learning rate: 1e-05
Batch size: 32
Gradient accumulation steps: 1
Image resolution: 768
Mixed-precision: None

More information on all the CLI arguments and the environment are available on your wandb run page.