Model Card for Model ID
This model performs multi-class semantic segmentation of gliomas (brain tumors) on 2D axial MRI slices using four modalities (T1, T1CE, T2, FLAIR). The model is optimized for research use based on the BraTS 2020 challenge data.
Model Details
Model Description
This is a deep learning model for medical image segmentation. It processes a 4-channel 128 X 128 image patch (from a single axial slice across four MRI modalities) and outputs a pixel-wise mask classifying regions into four classes: Background, Necrotic Core/Non-Enhancing Tumor (NCR/NET), Peritumoral Edema (ED), and Enhancing Tumor (ET).
Model Sources [optional]
Uses
The model's primary direct use is for brain tumor segmentation research, method comparison, and prototyping. It provides 2D segmentation masks when supplied with correctly preprocessed multi-modal MRI slices.
Out-of-Scope Use
Clinical Diagnosis or Treatment: This model must not be used for patient diagnosis, treatment planning, or any form of clinical decision-making. It is a research artifact.
Non-BraTS Data: Using the model on MRI data acquired with different protocols or without the required four co-registered modalities is likely to result in poor performance.
2D Segmentation: The model processes 2D slices independently. Its output must not be assumed to be a spatially smooth or coherent 3D segmentation mask without 3D post-processing.
Bias, Risks, and Limitations
Domain Specificity: Performance is tied to the characteristics of the multi-institutional BraTS dataset. Generalizing to different pathologies, scanners, or populations not represented in BraTS may lead to lower performance.
Modality Dependency: All four modalities (T1, T1CE, T2, FLAIR) are strictly required for inference.
Recommendations
Users (both direct and downstream) should be made aware that the model's output is an algorithmic prediction and must be validated by a domain expert (e.g., a neuroradiologist) if used in any clinical context (which is strongly discouraged).
How to Get Started with the Model
Use the code below to get started with the model.
[More Information Needed]
Training Details
Training Data
The model was trained on the BraTS 2020 dataset. This dataset comprises MRI scans of patients.
Data Characteristics: Each subject includes four co-registered modalities (T1, T1CE, T2, FLAIR) as NIfTI files (.nii.gz) and skull-stripped.
Annotations: Ground truth labels for tumor subregions (NCR/NET, ED, ET) were manually segmented by experts.
Training Procedure
Preprocessing [optional]
The training procedure relies on a multi-step preprocessing pipeline:
3D Volume Normalization: Z-score normalization applied per modality to the entire 3D volume.
Slicing: Axial slices were extracted.
Intensity Clipping: Values were clipped between the 2nd and 98th percentiles, then scaled to 0, 255.
Resizing: Images were resized from 240 X 240 to 128 X 128 (bilinear interpolation). Labels/masks were resized with Nearest-neighbor interpolation (order=0) to the same image size.
Global Normalization: Final per-tensor Z-score normalization applied to the 4-channel tensor.
Augmentation: Extensive geometric and intensity augmentations were applied:
- Geometric :
- HorizontalFlip (50% probability)
- VerticalFlip (50% probability)
- Rotate limit=15 degrees (50% probability)
- ShiftScaleRotate shift_limit=0.05, scale_limit=0.05, rotate_limit=15 (50% probability)
- Intensity :
- RandomBrightnessContrast brightness_limit=0.2, contrast_limit=0.2 (50% probability)
- GaussNoise (20% probability)
- Custom CLAHE (A.Lambda) (20% probability): Applies Contrast Limited Adaptive Histogram Equalization (clip_limit=2.0, tile_grid_size=(8, 8)) independently to each of the four MRI channels.
Training Hyperparameters
- Training regime: Mixed Precision (FP16/bf16 via torch.amp.autocast)
- Loss Function: Tversky Loss (alpha=0.3, beta=0.7, ignore_index=0 (background))
- Optimizer: AdamW lr:1e-5
- Batch Size: 32
- Epochs: 1000 (with Early Stopping)
Evaluation
Testing Data, Factors & Metrics
Testing Data
A separate, held-out test set from the BraTS 2020 data (10%) was used.
Factors
Evaluation was performed across four classes (0-3), with aggregated scores often reported for the three clinically relevant tumor subregions:
(Enhancing Tumor (ET): Class 3
Tumor Core (TC): Classes 1 and 3 (NCR/NET + ET)
Whole Tumor (WT): Classes 1, 2, and 3 (NCR/NET + ED + ET)
Metrics
- Dice Score
- IoU Score
- Precision
Results
Dice Score
| Subregion | Score |
|---|
| Whole Tumor (WT) | 0.8941 |
| Enhancing Tumor (ET) | 0.7864 |
| Tumor Core (TC) | 0.7569 |
IOU Score
| Subregion | Score |
|---|
| Whole Tumor (WT) | 0.8187 |
| Enhancing Tumor (ET) | 0.7106 |
| Tumor Core (TC) | 0.6900 |
Precision
| Subregion | Score |
|---|
| Whole Tumor (WT) | 0.8834 |
| Tumor Core (TC) | 0.8502 |
| Enhancing Tumor (ET) | 0.8296 |
Model Architecture
The model is a highly specialized, hybrid network designed for medical image segmentation. It uses a modified U-Net structure that integrates a powerful encoder with a sophisticated global context bottleneck.
| Component | Description | Rationale |
|---|
| Encoder | ResNeXt50_32x4d backbone, pre-trained on ImageNet. Initial Conv layer is adapted for 4 input channels (multimodality MRI). | Provides rich, deep, and hierarchically organized feature extraction. |
| Intermediate CNN | CNN blocks inserted after each ResNeXt layer block. | Further refines and regularizes deep feature maps before the next stage. |
| Transformer | Final encoder features are processed by Transformer Blocks with Positional Encoding. | Captures global, long-range dependencies across the feature map, mitigating the local view of pure CNNs. |
| ASPP | ASPP (Atrous Spatial Pyramid Pooling) module follows the Transformer. | Aggregates multi-scale features using dilated convolutions, essential for correctly segmenting tumor regions of various sizes. |
| Decoder | A standard upsampling path that reverses the encoder. | Restores the feature map resolution back to the original input size |
| Skip Connections | Attention Gates are used to connect encoder features to the decoder path. | Dynamically learns to weigh the importance of features in the skip connections, suppressing irrelevant background context and enhancing salient tumor features. |
| Final Layer | Convolutional layer. | Generates the final segmentation map with 4 output channels. |
Compute Infrastructure
[More Information Needed]
Hardware
[More Information Needed]
Software
Operating System: Linux (e.g., Ubuntu, CentOS)
Core Framework: PyTorch (version 1.10 or later recommended)
Essential Libraries:
- torch
- numpy
- nibabel
- albumentations
- scikit-image
- skimage.transform.resize
- pandas
- tqdm
- matplotlib
