You can deploy this model and start earning money today!

MesserMMP/
coronary-syntax-prediction

2407.19894

Views

No views yet

CardioSyntax v2: SYNTAX Score Prediction from Coronary Angiography

This repository contains code and pre-trained models for automated SYNTAX score prediction from coronary angiography videos. The SYNTAX score is a validated metric for quantifying the anatomical complexity of coronary artery disease.

Overview

CardioSyntax v2 is a large-scale dataset of 11,410 angiography studies with SYNTAX scores, designed for developing machine learning models for automated coronary lesion complexity assessment. This repository provides:

Dataset: 11,209 training studies (Philips Allura Clarity + Azurion 7) + 120 test studies (100 Philips + 20 Siemens) annotated by 6 independent interventional cardiologists
Baseline Models:
- Backbone network (R3D-18): single projection → SYNTAX contribution
- RNN head (LSTM/GRU/Transformer): multi-view aggregation → final SYNTAX score
Pre-trained Weights: 5-fold ensemble with post-calibration
Evaluation: Pearson correlation r ≈ 0.81, balanced accuracy ≈ 82% for SYNTAX > 22 classification

The dataset addresses inter-observer variability through multi-expert annotation and enables systematic investigation of domain shift effects across imaging systems.

Repository Structure

coronary-syntax-prediction/
├── backbone/                          # Single-frame backbone (3D ResNet)
│   ├── dataset.py                    # SyntaxDataset for single-view training
│   ├── pl_model.py                   # SyntaxLightningModule (Lightning wrapper)
│   ├── pl_train.py                   # Training script for backbone
│   └── __init__.py
├── full_model/                        # Multi-view RNN head + backbone
│   ├── rnn_dataset.py                # SyntaxDataset for multi-view (RNN) training
│   ├── rnn_model.py                  # SyntaxLightningModule with RNN variants
│   ├── rnn_train.py                  # Training script for full model
│   └── __init__.py
├── inference/                         # Inference & evaluation
│   ├── rnn_apply.py                  # Ensemble inference script
│   ├── metrics_visualization.py       # Plotly-based metrics & visualization
│   └── __init__.py
├── backbone_weights/                 # Backbone .pt checkpoints (5 folds × 2 arteries)
├── full_model_weights/               # Full model .pt checkpoints (5 folds × 2 arteries × variants)
├── scaling_coeffs/                   # Calibration coefficients (a, b) per fold
├── requirements.txt
└── README.md

Installation

bash

# Clone the repository
git clone https://huggingface.co/MesserMMP/coronary-syntax-prediction
cd coronary-syntax-prediction

# Install dependencies
pip install -r requirements.txt

Requirements

Key packages:

PyTorch 2.0+ with CUDA support
Lightning 2.x
torchvision, pytorchvideo
numpy, scikit-learn, pandas
plotly for visualization
click for CLI

See requirements.txt for full list.

Training

Stage 1: Backbone Training (Single-View)

Train 3D ResNet-18 on single angiographic projections (left/right coronary artery separately):

bash

python backbone/pl_train.py \
  -r /path/to/dataset_root \
  --fold 0 \
  -a right \
  --num-classes 2 \
  -b 50 \
  -f 32 \
  -v 256 256 \
  --max-epochs 10 \
  --num-workers 8 \
  --devices 0 \
  --precision bf16-mixed \
  --logdir ./logs/backbone \
  --use-weighted-sampler

Parameters:

-r, --dataset-root: Path to dataset root (default: .)
--fold: Fold number (0-4, default: 4)
-a, --artery: left or right (default: right)
-nc, --num-classes: Output units: 2 for (classification, regression) (default: 2)
-b, --batch-size: Batch size (default: 50)
-f, --frames-per-clip: Frames per clip (default: 32)
-v, --video-size: Frame resolution H W (default: 256 256)
--max-epochs: Epochs for full training (default: 10)
--num-workers: DataLoader workers (default: 8)
--devices: GPU device IDs (default: )
--precision: Training precision mode (default: bf16-mixed)
--logdir: Log directory (default: ./logs/backbone)
--use-weighted-sampler: Balance classes by score intervals (flag)
--seed: Random seed (default: 42)

Output: TensorBoard logs + .ckpt checkpoints in --logdir

Stage 2: RNN Head Training (Multi-View)

Train LSTM/GRU/Transformer aggregation head on multi-view studies:

bash

python full_model/rnn_train.py \
  -r /path/to/dataset_root \
  --fold 0 \
  -a right \
  --variant lstm_mean \
  --num-classes 2 \
  -b 8 \
  -f 32 \
  -v 256 256 \
  --max-epochs 15 \
  --num-workers 16 \
  --devices 0 \
  --precision bf16-mixed \
  --logdir ./logs/rnn \
  --backbone-pt-dir backbone_weights \
  --backbone-from-pt \
  --rnn-folds-dir rnn_folds

Key Parameters:

--variant: Head architecture:
- mean_out: Mean of projection scores (no RNN)
- mean: MLP on mean pooled backbone features
- lstm_mean: LSTM with mean pooling of hidden states
- lstm_last: LSTM, use last hidden state
- gru_mean, gru_last: GRU variants
- bert_mean, bert_cls, bert_cls2: Transformer encoder variants
- (default: lstm_mean)
--backbone-pt-dir: Path to .pt backbone weights (default: backbone_weights)
--backbone-from-pt: Load backbone from .pt files (flag, default: True)
--backbone-logdir: Alternative: load backbone from Lightning logs
--rnn-folds-dir: Directory with rnn_fold{fold}_train.json, rnn_fold{fold}_eval.json (default: rnn_folds)
--use-weighted-sampler: Balance by score (flag)
--pt-weights-format: If loading pre-trained head, expect .pt instead of .ckpt (flag)

Output: Full model checkpoints in --logdir/{artery}BinSyntax_R3D_fold{fold:02d}_{variant}_post/...

Inference

Run ensemble inference on test datasets:

bash

python inference/rnn_apply.py \
  -d "test_philips_100.json" "test_siemens_20.json" \
  -n "Philips Test" "Siemens Test" \
  -p "philips_100" "siemens_20" \
  -r /path/to/dataset_root \
  --model-dir full_model_weights \
  -v 256 256 \
  --frames-per-clip 32 \
  --num-workers 8 \
  --variant lstm_mean \
  --pt-weights-format \
  --use-scaling \
  --scaling-file scaling_coeffs.json \
  -e "Ensemble_lstm_mean" \
  -m metrics.json

Parameters:

-d, --dataset-paths: Multiple JSON test dataset paths (relative to --dataset-root)
-n, --dataset-names: Display names for each dataset (must match -d count)
-p, --postfixes: Suffixes for result files (must match -d count)
-r, --dataset-root: Dataset root (default: .)
--model-dir: Directory with full model .pt weights (default: full_model_weights)
-v, --video-size: Frame resolution H W (default: 256 256)
--frames-per-clip: Frames per clip (default: 32)
--num-workers: DataLoader workers (default: 8)
--variant: Head model variant (default: lstm_mean)
--pt-weights-format: Model weights are .pt raw state_dict (flag, default: True)
--use-scaling: Apply post-calibration scaling a*x+b from JSON (flag)
--scaling-file: Calibration coefficients file relative to --dataset-root
-e, --ensemble-name: Experiment name for metrics (e.g., "Ensemble_lstm_mean")
-m, --metrics-file: JSON output file path for all metrics
--seed: Random seed (default: 42)

Output:

Per-dataset predictions: results/{postfix}.json
Ensemble metrics: {metrics_file} with Pearson correlation, balanced accuracy, per-fold statistics
Plotly HTML plots: visualizations/{postfix}.html

Dataset

Link: https://huggingface.co/datasets/MesserMMP/coronary-angiography-syntax

Data Format

The dataset provides two different JSON annotation formats depending on the training stage:

Format 1: Backbone Training (Single-View/Per-Video)

Used for training the backbone model (backbone/ scripts). Each record represents one angiographic video projection.

File structure: folds/fold{fold_id}_{split}.json

json

[
  {
    "study_uid": "1.3.46.670589.28.26690171363123020190823114413193175",
    "series_uid": "1.3.46.670589.28.26690171363123020190823115102913895.2.2",
    "sop_uid": "1.3.46.670589.28.266901713631230201908231158145991852211512",
    "path": "../anon_data/chunk1/.../IM-2254-0039.dcm",
    "shape": [38, 512, 512],
    "artery": 0,
    "artery_prob": 1.9078343029832467e-05,
    "syntax": 0.0,
    "syntax_left": 0.0,
    "syntax_right": 0.0,
    "bin_syntax": 0,
    "manufacturer": "Philips",
    "device_model": "Allura Clarity"
  },
  {
    "study_uid": "1.3.46.670589.28.26690171363123020190823114413193175",
    "series_uid": "1.3.46.670589.28.26690171363123020190823115648342907.2.2",
    "sop_uid": "1.3.46.670589.28.266901713631230201908231158148282212211512",
    "path": "../anon_data/chunk1/.../IM-2260-0022.dcm",
    "shape": [15, 512, 512],
    "artery": 1,
    "artery_prob": 0.9559882879257202,
    "syntax": 0.0,
    "syntax_left": 0.0,
    "syntax_right": 0.0,
    "bin_syntax": 0,
    "manufacturer": "Philips",
    "device_model": "Allura Clarity"
  }
]

Field descriptions:

Field	Type	Description
`study_uid`	str	Unique study identifier (DICOM)
`series_uid`	str	Unique series identifier (DICOM)
`sop_uid`	str	Unique SOP instance UID (DICOM)
`path`	str	Relative path to DICOM video file
`shape`	list[int]	Video dimensions: [T, H, W] (frames, height, width)
`artery`	int	Coronary artery: 0 = left (LCA), 1 = right (RCA)
`artery_prob`	float	Model confidence for artery classification (0–1)
`syntax`	float	Total SYNTAX score (LCA + RCA)
`syntax_left`	float	SYNTAX score for left coronary artery
`syntax_right`	float	SYNTAX score for right coronary artery
`bin_syntax`	int	Binary classification: 0 = SYNTAX < threshold, 1 = SYNTAX ≥ threshold
`manufacturer`	str	Imaging equipment manufacturer (e.g., "Philips", "Siemens")
`device_model`	str	Equipment model (e.g., "Allura Clarity", "AXIOM-Artis")

Format 2: RNN Head Training (Multi-View/Per-Patient)

Used for training the full RNN model (full_model/ scripts). Each record represents one patient study with all angiographic projections grouped by artery.

File structure: rnn_folds/rnn_fold{fold_id}_{split}.json

json

[
  {
    "study_uid": "1.3.46.670589.28.26690171363123020190823114413193175",
    "manufacturer": "Philips",
    "device_model": "Allura Clarity",
    "syntax": 0.0,
    "syntax_left": 0.0,
    "syntax_right": 0.0,
    "bin_syntax": 0,
    "videos": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115102913895.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158145991852211512",
        "path": "../anon_data/chunk1/.../IM-2254-0039.dcm",
        "shape": [38, 512, 512],
        "artery": 0,
        "artery_prob": 1.9078343029832467e-05
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115648342907.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158148282212211512",
        "path": "../anon_data/chunk1/.../IM-2260-0022.dcm",
        "shape": [15, 512, 512],
        "artery": 1,
        "artery_prob": 0.9559882879257202
      }
    ],
    "videos_left": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115102913895.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158145991852211512",
        "path": "../anon_data/chunk1/.../IM-2254-0039.dcm",
        "shape": [38, 512, 512],
        "artery": 0,
        "artery_prob": 1.9078343029832467e-05
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115533239902.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158147562092211512",
        "path": "../anon_data/chunk1/.../IM-2258-0055.dcm",
        "shape": [50, 512, 512],
        "artery": 0,
        "artery_prob": 0.0003698925720527768
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115052517893.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158145581792211512",
        "path": "../anon_data/chunk1/.../IM-2253-0047.dcm",
        "shape": [47, 512, 512],
        "artery": 0,
        "artery_prob": 0.00018035581160802394
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115550658904.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158147942152211512",
        "path": "../anon_data/chunk1/.../IM-2259-0047.dcm",
        "shape": [41, 512, 512],
        "artery": 0,
        "artery_prob": 0.00022948876721784472
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115523054900.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158147162032211512",
        "path": "../anon_data/chunk1/.../IM-2257-0047.dcm",
        "shape": [43, 512, 512],
        "artery": 0,
        "artery_prob": 0.00034276593942195177
      }
    ],
    "videos_right": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115648342907.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158148282212211512",
        "path": "../anon_data/chunk1/.../IM-2260-0022.dcm",
        "shape": [15, 512, 512],
        "artery": 1,
        "artery_prob": 0.9559882879257202
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115657547909.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158149262272211512",
        "path": "../anon_data/chunk1/.../IM-2261-0070.dcm",
        "shape": [62, 512, 512],
        "artery": 1,
        "artery_prob": 0.9999938011169434
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115712699912.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158149652332211512",
        "path": "../anon_data/chunk1/.../IM-2262-0063.dcm",
        "shape": [54, 512, 512],
        "artery": 1,
        "artery_prob": 0.999855637550354
      }
    ],
    "videos_other": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115128658897.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158146391912211512",
        "path": "../anon_data/chunk1/.../IM-2255-0035.dcm",
        "shape": [33, 512, 512],
        "artery": 0,
        "artery_prob": 0.2866898477077484
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115132067898.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158146771972211512",
        "path": "../anon_data/chunk1/.../IM-2256-0035.dcm",
        "shape": [32, 512, 512],
        "artery": 0,
        "artery_prob": 0.3251875042915344
      }
    ]
  }
]

Field descriptions:

Field	Type	Description
`study_uid`	str	Unique study identifier (same across all videos for this patient)
`manufacturer`	str	Imaging equipment manufacturer
`device_model`	str	Equipment model
`syntax`	float	Total SYNTAX score (LCA + RCA)
`syntax_left`	float	SYNTAX score for left coronary artery
`syntax_right`	float	SYNTAX score for right coronary artery
`bin_syntax`	int	Binary classification label
`videos`	list[object]	All video projections (unfiltered)
`videos_left`	list[object]	Projections classified as left coronary (artery=0, high confidence)
`videos_right`	list[object]	Projections classified as right coronary (artery=1, high confidence)
`videos_other`	list[object]	Low-confidence projections (excluded from training); typically ~5–7% of total

Video object fields:

Field	Type	Description
`series_uid`	str	Unique series identifier (DICOM)
`sop_uid`	str	Unique SOP instance UID (DICOM)
`path`	str	Relative path to DICOM video file
`shape`	list[int]	Video dimensions: [T, H, W]
`artery`	int	Artery classification: 0 = left, 1 = right
`artery_prob`	float	Model confidence (0–1); threshold typically 0.5 for `videos_left`/`videos_right` inclusion

Test Set Format (Multi-Expert Annotations)

Test data (test_philips_100.json, test_siemens_20.json) follows Format 2 with additional expert annotations:

json

[
  {
    "study_uid": "...",
    "manufacturer": "Philips",
    "device_model": "Allura Clarity",
    "syntax": 18.5,
    "syntax_left": 10.0,
    "syntax_right": 8.5,
    "mean_syntax": 18.5,
    "expert0": 18,
    "expert1": 19,
    "expert2": 17,
    "expert3": 19,
    "expert4": 20,
    "expert5": 18,
    "videos_left": [...],
    "videos_right": [...],
    "videos_other": [...]
  }
]

Additional fields for test data:

Field	Type	Description
`mean_syntax`	float	Consensus SYNTAX score (mean of 6 experts)
`expert0`–`expert5`	float	Individual expert SYNTAX score (6 independent cardiologists)

Note: expert0 typically represents aggregated/consensus annotations, while expert1–expert5 are individual readers.

DICOM Video Format

Each DICOM file contains:

Dimensions: [T, H, W] = [frames, 512, 512] at 15 fps
Data type: uint8 (8-bit grayscale)
Frame count: Typically 15–60 frames per projection
Resolution: 512×512 pixels (standard for X-ray angiography)

Data Splits

Set	Files	Size	Equipment	Artery Classification
Backbone training	`folds/fold{0-4}_train.json`	~8,000 videos	Philips	Single video
Backbone validation	`folds/fold{0-4}_eval.json`	~1,000 videos	Philips	Single video
RNN training	`rnn_folds/rnn_fold{0-4}_train.json`	~2,200 studies	Philips	Multi-view (patient)
RNN validation	`rnn_folds/rnn_fold{0-4}_eval.json`	~290 studies	Philips	Multi-view (patient)
Test 1 (Philips)	`test_philips_100.json`	100 studies	Philips Allura Clarity	6 experts
Test 2 (Siemens)	`test_siemens_20.json`	20 studies	Siemens AXIOM-Artis	6 experts

SYNTAX > 0 prevalence: 39.26% in training data

Results

Baseline Model Performance (5-Fold Ensemble)

Metric	Test Set (Philips)	Test Set (Siemens)	Description
Pearson r	0.814	0.482	Raw predictions
Balanced Accuracy	0.687	0.566	Pre-calibration, SYNTAX > 22
Post-Calibration
Pearson r	0.816	0.482	After scaling
Balanced Accuracy	0.825	0.643	After scaling, SYNTAX > 22

Key findings:

Strong performance on in-distribution (Philips) test data
Domain shift effect on out-of-distribution (Siemens) data
Calibration effective for Philips, limited benefit for Siemens
Inter-observer variability constrains upper bound on accuracy

Model Architecture

Backbone: R3D-18 (video ResNet) pre-trained on Kinetics-400
Head (default): LSTM with mean-pooling aggregation + linear regressor
Output: 2 channels (classification logit + log-transformed regression)
Loss: Weighted BCE (classification) + scaled MSE (regression)

Weights & Checkpoints

Pre-trained weights available in repository:

backbone_weights/: .pt files for 5 folds × 2 arteries
- Example: RightBinSyntax_R3D_full_fold00.pt
full_model_weights/: .pt files for ensemble variants
- Example: RightBinSyntax_R3D_fold00_lstm_mean_post_best.pt
scaling_coeffs/: Calibration parameters (a, b) per fold

Citation

If you use this dataset or code, please cite:

bibtex

@dataset{cardiosyntax_v2_2025,
  title={CardioSyntax v2: Angiographic dataset for SYNTAX score estimation},
  author={MesserMMP and collaborators},
  year={2025},
  url={https://huggingface.co/datasets/MesserMMP/coronary-angiography-syntax}
}

Related Work

SYNTAX Score Definition: Sianos et al., EuroIntervention (2005)
Original CardioSyntax: Ponomarchuk et al., arXiv:2407.19894 (2024)
Video Understanding: Hara et al., CVPR (2018) — R3D architecture

License

CC0 1.0 (Public Domain)

Contact & Support

For issues, questions, or contributions, please refer to the HuggingFace repository discussions.

Last updated: January 2026