Patient Timeline Forecasting: Future Conditions
Decoder-only Transformer trained on tokenized Synthea patient timelines. The model is pretrained using a next-event prediction objective and fine-tuned to predict which target conditions are newly diagnosed during the 5-year forecasting window following each patient's anchor date.
The accompanying GitHub repository contains the full data-processing, training, evaluation, and inference pipeline.
Model
- Architecture: Decoder-only Transformer
- Task: 40-condition multi-label classification
- Input: Tokenized longitudinal patient timelines
- Pretraining: Autoregressive next-event prediction
- Fine-tuning: Supervised multi-label classification
- Pooling: Attention pooling
- Embedding dimension: 256
- Transformer layers: 6
- Attention heads: 8
- Maximum sequence length: 1024
Files
finetune_best.pt — final selected model weightsvocab.json — event-token vocabulary fitted on the training split onlylab_bins.json — per-lab-code quantile bin edges fitted on the training split onlytarget_codes.json — ordered target condition codes predicted by the classification headconfig.yaml — configuration used to produce the checkpoint
Training and Model Selection
The final checkpoint was produced using the selected training configurations tracked in Weights & Biases.
- Pretraining run:
final_training
- Fine-tuning run:
final_finetuning
- Checkpoint:
finetune_best.pt
- Model selection: Best fine-tuning checkpoint selected based on validation macro-AUROC.
Validation Results
Because test-set outcomes are withheld, the reported metrics are from the held-out validation cohort.
| Metric | Score |
|---|
| Macro-AUROC | 0.7288 |
| mAP | 0.2000 |
Reproducibility
The vocabulary and laboratory quantization boundaries are derived exclusively from the training split to prevent distribution leakage.
The config.yaml, vocab.json, lab_bins.json, and target_codes.json files are provided alongside the checkpoint to support reproducible inference.
Source Code
Full source code, training instructions, evaluation procedures, and the technical report are available in the accompanying GitHub repository.
Intended Use and Limitations
This model is provided for research and educational purposes. It is not intended for clinical diagnosis or treatment decisions.
Performance is reported on a held-out validation cohort because final test outcomes are withheld. Performance may vary across target conditions, particularly for rare outcomes.
