Diffusion model trained on a public dataset of images from image data resource to create highly detailed accurate depictions of flourescent and super-resolution cell images.
image
Ground-truth image data obtained from idr:
realsamples
py
1from diffusers import DDIMPipeline, DDPMPipeline, PNDMPipeline
23model_id ="nakajimayoshi/ddpm-iris-256"45# load model and scheduler6ddim = DDIMPipeline.from_pretrained(model_id)# you can replace DDPMPipeline with DDIMPipeline or PNDMPipeline for faster inference78# run pipeline in inference (sample random noise and denoise)9image = ddim().images[0]1011# save image12image.save("sample.png")
The role of generative AI in the science is a new discussion and the merits of it have yet to be evaluated. Whilst current image-to-image and text-to-image models make it easier than ever to create stunning images, they lack the specific training sets to replicate accurate and detailed images found in flourescent cell microscopy.
We propose ddpm-IRIS, a difusion network leveraging Google's Diffusion Model to generate visual depitctions of cell features with more detail than traditional models.
Hyperparameters:
image_size = 256
train_batch_size = 16
eval_batch_size = 16
num_epochs = 50
gradient_accumulation_steps = 1
learning_rate = 1e-4
lr_warmup_steps = 500
save_image_epochs = 10
save_model_epochs = 30
mixed_precision = 'fp16'
trained on 1 Nvidia A100 40GB GPU over 50 epochs for 2.5 hours.
