REVEAL++
REVEAL++: Differentiable Phenotypic Grouping for Vision–Language Retinal Modeling of Alzheimer's Disease Risk
REVEAL++ is a multimodal retinal vision-language model for learning representations from retinal fundus images and structured clinical risk narratives. The model was developed for research on incident Alzheimer's disease risk prediction using UK Biobank retinal imaging data.
Instead of assigning subjects to fixed phenotypic groups, REVEAL++ uses continuous, differentiable phenotypic weighting to model inter-subject similarity. This allows partially similar subjects to contribute proportionally during multimodal contrastive learning, better reflecting the heterogeneous and spectrum-like nature of neurodegenerative disease risk.
Model Description
REVEAL++ aligns retinal fundus images with clinical risk narratives in a shared representation space. The model builds on the REVEAL framework and replaces hard group-aware contrastive learning with a continuous phenotypic-weighted objective.
The framework uses:
- A retinal image encoder based on RETFound.
- A biomedical text encoder based on GatorTron.
- Projection heads that map image and text embeddings into a shared multimodal space.
- Soft inter-subject phenotypic weights for graded multi-positive contrastive learning.
- Downstream evaluation using learned representations for incident Alzheimer's disease prediction.
Available checkpoint files:
| Checkpoint | Description |
|---|
revpp.pt | REVEAL++ checkpoint |
gacl_true.pt | REVEAL baseline with hard group-aware contrastive learning |
gacl_false.pt | REVEAL baseline without group-aware contrastive learning |
Intended Use
This model is intended for research use only.
Appropriate uses include:
- Multimodal representation learning with retinal images and clinical narratives.
- Research on retinal biomarkers and neurodegenerative disease risk.
- Benchmarking retinal vision-language models.
- Downstream feature extraction for incident Alzheimer's disease prediction.
- Reproducing or extending the REVEAL++ experimental framework.
This model is not intended for clinical diagnosis, clinical decision-making, treatment planning, or deployment in patient-facing settings.
Limitations
REVEAL++ has several important limitations:
- The model was evaluated on UK Biobank data, and performance may not generalize to other populations, imaging devices, clinical settings, or datasets.
- The downstream task is incident Alzheimer's disease prediction, which is a difficult, low-prevalence, long-horizon risk prediction problem.
- The model should not be interpreted as a diagnostic tool.
- The model may inherit biases from the underlying UK Biobank cohort and from the pretrained RETFound and GatorTron encoders.
- Raw UK Biobank retinal images, clinical variables, and protected health information are not included with this repository.
- External validation is needed before considering any translational or clinical application.
- The released checkpoints are intended to support reproducibility and research, not medical use.
Training Dataset
The model was trained using UK Biobank retinal fundus imaging data paired with structured clinical risk narratives.
The training data consist of:
- Retinal fundus images.
- Clinical risk narratives derived from structured questionnaire and clinical variables.
- COCO-style JSON annotation files containing image metadata and clinical text captions.
- Retinal image feature files used during training.
Expected data files include:
1data/UKB/captions_train.json
2data/UKB/captions_val.json
3data/UKB/captions_test2.json
4data/UKB/Macular_Measurement.csv
