HNE2Cell — H&E Whole-Slide Image Cell Detection & Classification
HNE2Cell detects and classifies 15 cell types (16 output classes including background) from H&E-stained whole-slide images (WSI).
It takes 256×256 px patches as input and outputs per-cell contours, centroids, and type labels.
Output Classes (background + 15 cell types)
ID Cell Type Color 0 Background — 1 Malignant 🔴 Red 2 CD4 T 🔵 Dodger Blue 3 CD8 T 🔵 Royal Blue 4 B 🔵 Blue 5 Plasma 🔵 Cornflower Blue 6 Macrophage 🔵 Powder Blue 7 Myeloid 🔵 Steel Blue 8 DC 🔵 Deep Sky Blue 9 Fibroblast 🟢 Forest Green 10 Endothelial 🟢 Medium Sea Green 11 Pericyte 🟢 Lime Green 12 Epithelial 🟠 Dark Orange 13 Immune_Other ⚪ Light Blue 14 Stromal_Other 🟤 Olive Drab 15 Dead ⚫ Grey
Pipeline Overview
The full pipeline consists of three steps:
┌───────────────┐ ┌────────────────┐ ┌──────────────────┐
│ 1. Normalize │ ──→ │ 2. Patchify │ ──→ │ 3. Inference │
│ (Reinhard) │ │ (256px, 64ov) │ │ (Cell Detection)│
└───────────────┘ └────────────────┘ └──────────────────┘
SVS / TIF PNG patches Masks + CentroidsMagnification
40x (Recommended) 20x (Supported) Accuracy Best — fine-grained cell boundaries Good — may miss small immune cells Speed More patches per slide Fewer patches, faster Use when Immune cell subtyping matters Quick screening / large cohorts
40x is strongly recommended. The model was primarily trained on 40x data. 20x works but expect reduced precision for small cells (lymphocytes, DCs).
System Requirements
Software dependencies (tested versions)
Core packages (as reported in the manuscript):
Python 3.10
pytorch == 2.5.1
timm == 1.0.8
transformers == 4.44.0
scanpy == 1.10.3
squidpy == 1.5.0
spatialdata == 0.2.5
scikit-image == 0.24.0
scikit-learn == 1.2.2
scipy == 1.13.1
shapely == 2.0.7
Additional utilities required by the pipeline scripts:
torchvision (matching the PyTorch 2.5.1 release)
tifffile, Pillow, opencv-python-headless, pandas, tqdm
huggingface_hub
openslide-python (optional, for .svs files)
Operating systems tested
Ubuntu 22.04 LTS
Ubuntu 20.04 LTS
(Not tested on Windows/macOS.)
Hardware requirements
Note: WSI processing is memory-intensive. This pipeline is designed for
server- or workstation-class hardware, not standard desktops.
Minimum (small WSIs, ~1–2 GB):
GPU: NVIDIA GPU with ≥12 GB VRAM
RAM: 32 GB (64 GB strongly recommended)
Disk: 100 GB free
Recommended (typical WSIs, 2–10 GB):
GPU: NVIDIA A100 / RTX 4090 / RTX 3090 (≥24 GB VRAM)
RAM: ≥128 GB
Disk: 500 GB+ free (intermediate Aligned-hne.tif can be 20–50 GB per slide)
Tested configurations:
NVIDIA A100 (40 GB VRAM), 256 GB RAM, Ubuntu 22.04
NVIDIA RTX 3060 (12 GB VRAM), 64 GB RAM, Ubuntu 22.04
CPU-only inference is not supported in practice — full WSI inference would take
days even on a high-core-count CPU.
Installation Guide
Recommended: Conda environment from cellvit_rv3.yml
The repository includes a frozen conda environment file with all dependencies pinned
to the exact versions used in the manuscript.
1 # 1. Download environment file
2 wget https://huggingface.co/roobee79/HNE2Cell/resolve/main/cellvit_rv3.yml
3
4 # 2. Create environment
5 conda env create -f cellvit_rv3.yml
6
7 # 3. Activate
8 conda activate cellvit_rv3
Typical install time: ~10–15 minutes on a Linux server with a stable network connection
(dominated by the PyTorch + CUDA toolkit download).
Download the model
1 from huggingface_hub import hf_hub_download
2
3 model_path = hf_hub_download (
4 repo_id = "roobee79/HNE2Cell" ,
5 filename = "HNE2cell_pub_patch73_jit.pt"
6 )
Demo: Reproducible Walkthrough
To verify your installation, run the pipeline on the example slide included in this
repository (TCGA-56-8628-01Z-00-DX1, LUSC, ~36 MB).
Download the model, example slide, and reference image
1 from huggingface_hub import hf_hub_download
2
3 REPO_ID = "roobee79/HNE2Cell"
4
5 config_path = hf_hub_download ( repo_id = REPO_ID , filename = "config.json" )
6
7 model_path = hf_hub_download (
8 repo_id = REPO_ID ,
9 filename = "HNE2cell_pub_patch73_jit.pt"
10 )
11
12 slide_path = hf_hub_download (
13 repo_id = REPO_ID ,
14 filename = "TCGA-56-8628-01Z-00-DX1.AAC57164-E0F9-4DF0-87EA-5C50FB201895.svs"
15 )
16
17 ref_path = hf_hub_download (
18 repo_id = REPO_ID ,
19 filename = "standard-ilc.tif"
20 )
Run the pipeline
1 # Place the downloaded slide in a working directory
2 mkdir -p example/slides
3 cp < slide_path > example/slides/
4
5 # Step 1: Normalize
6 python normalize.py \
7 --input_dir ./example/slides \
8 --target ./standard-ilc.tif
9
10 # Step 2: Patchify at 40x
11 python patchify.py \
12 --input_dir ./example/slides \
13 --magnification 40 \
14 --patch_size 256 \
15 --overlap 64 \
16 --workers 8
17
18 # Step 3: Inference
19 python inference.py \
20 --input_dir ./example/slides/TCGA-56-8628-01Z-00-DX1 .. ./patches \
21 --output_dir ./example/results \
22 --model_path ./HNE2cell_pub_patch73_jit.pt \
23 --magnification 40 \
24 --batch_size 32
Expected output
example/results/
├── Aligned-hne.tif # Normalized full-resolution H&E
├── Aligned-hne.jpg # 4× preview
├── patch_*_mask.png # Per-patch cell type masks
└── patch_*_centroid.csv # Cell centroids with type labelsExpected results on the example slide (TCGA-56-8628-01Z-00-DX1):
Approximately 63,000 cells are detected across the 16 classes.
Small variation (±a few percent) is expected between hardware configurations.
Expected runtime
Hardware Full pipeline runtime NVIDIA A100 (40 GB) + 256 GB RAM ~20 min NVIDIA RTX 3060 (12 GB) + 64 GB RAM ~30 min
A system without sufficient RAM (<32 GB) will fail at the normalization step
due to full-resolution image loading.
The example slide is from
TCGA-LUSC and is redistributed under the
NIH Genomic Data Sharing Policy .
Instructions for Use (On Your Own Data)
1 # Step 1: Color normalization (Reinhard method)
2 python normalize.py \
3 --input_dir /path/to/slides \
4 --target /path/to/standard-ilc.tif
5
6 # Step 2: Extract patches (40x recommended)
7 python patchify.py \
8 --input_dir /path/to/slides \
9 --magnification 40 \
10 --patch_size 256 \
11 --overlap 64 \
12 --workers 8
13
14 # Step 3: Cell detection & classification
15 python inference.py \
16 --input_dir /path/to/patch_folders \
17 --output_dir /path/to/results \
18 --model_path ./HNE2cell_pub_patch73_jit.pt \
19 --magnification 40 \
20 --batch_size 32
Input / Output Details
Input
Step Input Format Normalize Raw WSI .svs, .tif, .tiff, .ndpiPatchify Normalized image Aligned-hne.tif (from Step 1)Inference Patches 256×256 px PNG files
Output
File Description Aligned-hne.tifFull-resolution normalized H&E image Aligned-hne.jpg4× downsampled preview recon.tifTissue-only reconstruction (intermediate) *_mask.pngPer-patch cell segmentation mask (colored by type) *_centroid.csvCell centroids with columns: slide_id, x, y, celltype, celltype_name
Centroid CSV format
1 slide_id , x , y , celltype , celltype_name
2 patch_0_0 , 112.3 , 87.5 , 1 , Malignant
3 patch_0_0 , 45.1 , 201.2 , 2 , CD4T
4 ...
To convert patch-local coordinates to WSI-global coordinates:
1 # Parse patch filename: {prefix}_{x_offset}_{y_offset}.png
2 x_global = x + x_offset
3 y_global = y + y_offset
Normalization Reference Image
The pipeline uses Reinhard color normalization in LAB color space. You need a reference image
(standard-ilc.tif) that represents your target stain appearance. The reference image is
included in this repository, or you can supply your own.
File Structure
HNE2Cell/
├── README.md # This file
├── HNE2cell_pub_patch73_jit.pt # TorchScript model
├── normalize.py # Step 1: Reinhard normalization
├── patchify.py # Step 2: Patch extraction
├── inference.py # Step 3: Model inference
├── post_processing.py # Cell post-processing module
├── tools.py # Utility functions
├── standard-ilc.tif # Reference image for normalization
└── TCGA-56-8628-01Z-00-DX1.AAC57164-E0F9-4DF0-87EA-5C50FB201895.svs # Example slide (TCGA-LUSC)Citation
If you use HNE2Cell in your research, please cite:
1 @misc{hne2cell,
2 title={Spatial transcriptomics–supervised deep learning enables single-cell mapping of tumor immune architecture from routine histology},
3 year={2026},
4 url={https://huggingface.co/roobee79/HNE2Cell}
5 }
The example slide is derived from data generated by the TCGA:
https://portal.gdc.cancer.gov/ .
License
This repository uses a dual licensing scheme commonly adopted in academic ML/biomedical projects (e.g., SAM, LLaMA):
Source code (.py files): Released under the MIT License . See LICENSE.
Model weights (HNE2cell_pub_patch73_jit.pt): Released under
Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) .
See MODEL_LICENSE. Free for academic and non-commercial research use.
For commercial licensing, please contact:
Ewha University-Industry Collaboration Foundation
Technology Commercialization Team
Ewha Womans University, 52 Ewhayeodae-gil, Seodaemun-gu, Seoul, Republic of Korea
Web:
https://research.ewha.ac.kr |
https://epm.ewha.ac.kr
Example slide (
TCGA-56-8628-*.svs): Derived from TCGA-LUSC, governed by
the
NIH Genomic Data Sharing Policy .
