Custom fine-tuned version of NVIDIA's segformer model for colony slides in microscopy.
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021). SegFormer: Simple and efficient design for semantic segmentation with transformers. arXiv preprint arXiv:2105.15203. https://arxiv.org/abs/2105.15203
This model takes 512x512 images for segmentation.
The Segformer-Cityscapes model was changed to a ternary classifier and fine-tuned on custom training data, where organoids and necrosis were made as separate masks and then merged with different grayscale values.
test12.png
Mask_test12.png
Mask2_test12.png
After training, it was able to correctly identify organoids and necrosis.
example2.png
example.png
The python program (see linked GitHub) then uses the masks to annotate the images and provide statistics about the colonies.
