HeteroTissueDiffuse is a latent diffusion model (LDM) that synthesizes heterogeneous histopathology images by conditioning on both a binary semantic map and raw tissue crop exemplars. Unlike text- or embedding-guided approaches, it injects actual tissue appearance directly into the diffusion process, preserving staining characteristics, nuclear morphology, and cellular texture.
The model addresses a fundamental limitation of prior generative methods in histopathology: the tendency to produce homogeneous (single-tissue-type) images. By providing spatially-registered visual prompts for each tissue class, the model generates realistic heterogeneous slides that accurately reflect real-world tissue organization.
Architecture
Base: CompVis Latent Diffusion Model with VQ-regularized autoencoder
First stage: VQModelInterface (3-channel latent, 8192 codebook)
Conditioning encoder: SpatialRescaler with in_channels=8 (replaces ADE20K default of 182)
U-Net: 128 base channels, attention at resolutions 32/16/8
Image size: 256×256 pixels
Sampling: DDIM, 200 steps, η=1
8-Channel Conditioning Tensor
Channel 0: normal onehot mask (1 where segmentation == 0)
Channels 1–3: normal tissue crop RGB (float32, normalized to [-1,1])
Channel 4: tumor onehot mask (1 where segmentation == 1)
Channels 5–7: tumor tissue crop RGB (float32, normalized to [-1,1])
The tissue crops are small patches (typically 30–60px) extracted from a reference slide and pasted spatially within the corresponding mask region. This lets users control staining appearance at inference time without any fine-tuning.
prompt_frame_XXX.png — visualization of the conditioning (mask + overlaid crops)
Performance
Downstream Segmentation (IoU)
Training data
Camelyon16
PANDA
Real images
0.72
0.96
Synthetic (ours)
0.71
0.95
Synthetic (no conditioning)
0.51
0.82
Pathologist Assessment
A certified pathologist evaluated 120 images in a blinded study. Synthetic images conditioned with visual prompts received quality scores indistinguishable from real images:
"The generated images tended to have equal or higher quality than the real images."
Intended Use
Research: generating large annotated synthetic histopathology datasets for downstream model training
Augmentation: expanding small annotated datasets with realistic diverse tissue variations
Privacy-preserving data sharing: synthetic data as a substitute for patient slides
Patches from 11,765 TCGA whole-slide images were embedded using a histopathology foundation model (PathDino), then clustered into 100 tissue phenotypes via k-means. These clusters form pseudo-semantic maps for training without manual annotation.
Citation
bibtex
1@InProceedings{Alfasly2025HeteroTissueDiffuse,
2 author = {Alfasly, Saghir and Uegami, Wataru and Hoq, MD Enamul and Alabtah, Ghazal and Tizhoosh, H.R.},
3 title = {Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology},
4 booktitle = {Neural Information Processing Systems (NeurIPS)},
5 month = {December},
6 year = {2025}
7}
License
This model is released under the CreativeML Open RAIL-M license, inherited from CompVis/stable-diffusion. This license permits research and commercial use but prohibits use cases that cause harm (e.g., generating deceptive or malicious content). See the full license here.
