🧠 Glioma Survival Transformer — Multimodal Longitudinal MRI + Clinical

Model Description

1. Time-to-progression (continuous hazard via Cox PH)
2. 6-month and 12-month progression probability (binary classification)

Architecture

Input: Longitudinal MRI (T×5×64×64×64) + Clinical PCA features (~20-40 dims)
    │
    ├── 3D CNN Encoder (4 blocks: 32→64→128→256 channels, GroupNorm, ReLU, MaxPool3d)
    │     → AdaptiveAvgPool3d(1) → 256-dim per timepoint
    │     + append [time_gap_months, log(tumor_volume)] → 258-dim
    │
    ├── Input Projection (Linear 258→256, needed for nhead=4 divisibility)
    │
    ├── Temporal Transformer Encoder (2 layers, 4 heads, dim_ff=512)
    │     ★ CAUSAL MASKING — each timepoint only attends to itself + past
    │     ★ Learnable [CLS] token prepended → 256-dim temporal summary
    │
    ├── Clinical MLP (PCA_dim → 128 → ReLU → LayerNorm → Dropout(0.3) → 64 → ReLU)
    │     → 64-dim
    │
    └── Fusion (Concat 256+64=320 → FC(128) → ReLU → Dropout → FC(64) → ReLU)
          ├── Hazard Head: FC(64→1), linear — for Cox PH loss
          └── BCE Head: FC(64→1) + Sigmoid — auxiliary 12-month classification

• Causal masking in Transformer prevents future information leakage
• GroupNorm (not BatchNorm) for small batch sizes (BS=2)
• Lightweight 3D CNN (not ResNet-18) to fit on T4 GPU with 64³ crops
• Dual loss: Cox PH partial likelihood + 0.1 × masked BCE
• ~2.35M parameters — intentionally small for the dataset size

Dataset

1. MRI imaging dataset: Longitudinal NIfTI volumes (.nii.gz) from TCIA
2. Clinical tabular dataset: Demographics, genomics, treatment, survival outcomes (.xlsx)

Training Procedure

• Cross-validation: 5-fold StratifiedGroupKFold (grouped by Patient_ID, stratified on progression event)
• Optimizer: AdamW (lr=1e-4, weight_decay=1e-2)
• Scheduler: ReduceLROnPlateau (patience=5, factor=0.5) watching val C-index
• Batch: 2 × 4 gradient accumulation = effective batch size 8
• Mixed precision: FP16 via torch.amp
• Early stopping: Patience 15 on validation C-index
• Gradient clipping: max_norm=1.0
• Augmentation (train only): Random flips (all 3 axes, p=0.5), intensity scaling (±10%), Gaussian noise (σ=0.1)

• StandardScaler + PCA (95% variance) fit strictly inside each CV fold — no data leakage
• Censored patients (PFS < horizon but no event) excluded from BCE loss via binary mask
• Weighted random sampling to address class imbalance on 12-month labels
• Patients without MRI excluded entirely from multimodal model (not zeroed out)

MRI Preprocessing Pipeline

1. Load .nii.gz via nibabel → float32
2. Reorient to RAS canonical
3. Z-score normalize per modality using only non-zero brain voxels: vol[mask] = (vol[mask] - mean) / (std + 1e-8)
4. Crop to tumor bounding box from segmentation + 16-voxel margin
5. Resize to 64×64×64 using SimpleITK BSpline (images) / NearestNeighbor (masks)
6. Stack 4 modalities + mask → shape (5, 64, 64, 64)
7. Compute tumor sub-region volumes: NETC (1), SNFH (2), ET (3), RC (4), Total

Censoring Logic

Loss Function

L = L_cox + 0.1 × L_bce

L_cox = -mean[ h_i - log( Σ_{j: t_j ≥ t_i} exp(h_j) ) ]   (only uncensored events)
L_bce = BCE(risk_prob, label_12mo)                            (only non-censored patients)

Evaluation Metrics

Evaluation Plots (saved to outputs/plots/)

1. Kaplan-Meier curves — predicted high-risk vs low-risk (median split), log-rank p-value
2. ROC curves — 6-month and 12-month horizons
3. Training curves — loss and C-index per epoch, all folds overlaid
4. Calibration curve — predicted probability vs observed frequency at 12 months
5. Tumor volume trajectories — top-4 patients, all sub-regions across timepoints

Files in this Repository

How to Use

Google Colab (recommended)

1. Open Glioma_Survival_Pipeline.ipynb in Colab
2. Set runtime to GPU (T4 or A100)
3. Place MU-Glioma-Post data at /content/drive/MyDrive/BrainTumor/
4. Run all cells top-to-bottom
5. Without real data, the notebook auto-generates synthetic data for smoke-testing

Checkpoint Loading

Limitations

• Small sample size: Only ~92 patients with complete multimodal data. High variance.
• Single institution: MU-Glioma-Post is single-center. No external validation.
• No external test set: All metrics from cross-validation only.
• High censoring rate: Affects reliability of survival estimates at longer horizons.
• Missing modalities: Some timepoints have missing MRI sequences (zero-filled).

Intended Use

• ✅ Academic research in neuro-oncology and medical AI
• ✅ Hypothesis generation about multimodal features and glioma progression
• ✅ Baseline for comparison with more advanced architectures

Out-of-Scope Use

• ❌ Clinical diagnosis or treatment planning
• ❌ Individual patient prognostication without clinical oversight
• ❌ Application to non-glioma brain tumors without retraining
• ❌ Use with different MRI protocols/scanners without domain adaptation

Citation

References

1. Jang, B.S. et al. (2018). Prediction of pseudoprogression versus true progression using machine learning and MRI. Scientific Reports.
2. Pérez-García, F. et al. (2021). LUMIERE: Longitudinal glioblastoma MRI evaluation. NeuroImage: Clinical.
3. Cox, D.R. (1972). Regression models and life-tables. JRSS-B.
4. Lee, C. et al. (2018). DeepHit: A deep learning approach to survival analysis with competing risks. AAAI.
5. MU-Glioma-Post. The Cancer Imaging Archive (TCIA). https://www.cancerimagingarchive.net/collection/mu-glioma-post/