This repository hosts the backbone weights for DALE-CT-1S-v2 (Depth-Aware Latent-Euclidean Computed Tomography), a foundational Vision Transformer (ViT-Large) trained from scratch on chest CT with the LeJEPA objective plus a single-source (1S) dense auxiliary supervision head over the 118 TotalSegmentator anatomical classes, applied to both the [CLS] token and the patch tokens. The ReXGroundingCT abnormality head is held inert, so this model isolates the contribution of anatomical supervision against DALE-CT-2S.

This is the single-source configuration benchmarked in the DALE-CT paper. It supersedes the earlier DALE-CT-1S (patch-14, [CLS]-only supervision), which remains available as the backbone used by Ker-VLJEPA-3B.

Quick Load (timm)

python

import timm
model = timm.create_model("hf-hub:Kentucky-Open-Science/DALE-CT-1S-v2", pretrained=True)
model.eval()

Inputs must be Hounsfield-Unit slices preprocessed exactly as during training (clipping + z-score; see the full example below).

The DALE-CT Family

All numbers are our own head-to-head measurements: every model (including the public 3D baselines in the paper) is probed under one linear-probing MIL protocol on shared splits (CT-RATE n = 992 test scans; RAD-ChestCT n = 360). See the paper for the full protocol and confidence intervals.

Model	CT-RATE Macro AUROC	RAD-ChestCT AUROC (frozen / retrained probe)	Role
DALE-CT-0-L ⭐	0.8156	0.6281 / 0.7572	Recommended general-purpose backbone — best 2D external-transfer point estimates; supervision-free at ~287k-scan scale
DALE-CT-2S	0.8247	0.6252 / 0.7389	Best in-domain (CT-RATE)
DALE-CT-1S-v2	0.8098	0.6284 / 0.7334	Anatomical (TotalSegmentator) dense supervision only
DALE-CT-0	0.8057	0.5946 / 0.7477	Pure self-supervised, CT-RATE
Finetuned DINOv2	0.7953	0.6252 / 0.7550	Continual-pretraining baseline — strongest dense (patch-level) features

Paper: DALE-CT: Depth-Aware 2D Slice Encoders Learn an Anatomical World Model of Chest CT · Code: Kentucky-Open-Science/DALE-CT · Benchmark: Kentucky-Open-Science/chest-ct-foundation-model-benchmark

Model Details

Model Type: Vision Transformer (ViT-Large) for chest CT analysis.
Developed by: Institute for Biomedical Informatics Center for Applied AI (IBI-CAAI), University of Kentucky
Base Model Architecture: vit_large_patch14_dinov2 (via timm), randomly initialized and trained from scratch with patch_size=16, img_size=512, in_chans=1, dynamic_img_size=True.
Input: 1-channel grayscale CT slice (Hounsfield Units, preprocessed as below).
Output: class token and patch tokens (embedding dimension 1024).
License: CC BY-NC-SA 4.0 (inherited from the CT-RATE dataset terms).

Training Data

Dataset: train split of CT-RATE (25,692 chest CTs).
Auxiliary labels: TotalSegmentator masks (118 classes, auto-generated) as soft fractional-coverage targets on [CLS] and patch tokens. ReXGroundingCT is inert for this variant.
Preprocessing: HU clipped to [-997.0, 888.0], mapped to [0, 1], then z-score normalized (dataset mean -142.39, std 360.97 in HU space).

Training Procedure

DDP, bf16, 8×H100 GPUs; 66,667 iterations at global batch 384 (48/GPU), 6,667-step warmup, peak LR 3.0e-4 decaying to 3.0e-5 — budget-matched to DALE-CT-0/2S (25.6M images), so the 0 → 1S-v2 → 2S ladder is architecture-, patch-size-, and budget-controlled.
Depth-aware multi-crop: two global 256² crops from the slab center slice and eight local 144² crops sampled across a continuous 12 mm physical slab, with label-guided local crops (p=0.8).
Objective: L_LeJEPA + 0.1 · L_Aux, where L_Aux is BCE-with-logits over TotalSegmentator soft-coverage targets on [CLS] and patch tokens, averaged over global/local crops.

Preprocessing Example

python

import torch, numpy as np, timm

model = timm.create_model("hf-hub:Kentucky-Open-Science/DALE-CT-1S-v2", pretrained=True)
model.eval()

clip_min, clip_max, mean_hu, std_hu = -997.0, 888.0, -142.39, 360.97
rng = clip_max - clip_min
norm_mean, norm_std = (mean_hu - clip_min) / rng, std_hu / rng

hu_slice = np.random.uniform(-1000, 1000, size=(512, 512))  # replace with real HU data
x = torch.from_numpy(hu_slice).float().clamp(clip_min, clip_max)
x = ((x - clip_min) / rng - norm_mean) / norm_std
x = x[None, None]  # (1, 1, H, W)

with torch.no_grad():
    cls_feature = model(x)                    # (1, 1024)
    tokens = model.forward_features(x)        # (1, 1 + N_patches, 1024)

Citation

If you use this model, please cite the DALE-CT paper (https://arxiv.org/abs/2606.07775).