You can deploy this model and start earning money today!

dmolino/
text2ct-weights

Name: text2ct-weights
Availability: InStock

LDM+VAE

medical

text-to-3d

ibrahimhamamci/CT-RATE

dmolino/CT-RATE_Generated_Scans

2506.00633

apache-2.0

Views

No views yet

Text-to-CT Model Weights

Checkpoints for "From Alignment to Synthesis: Contrastive Volumetric Grounding for Text-to-CT Generation" (Molino et al., BMVC 2026).

Note on versions. The checkpoints currently hosted here correspond to the earlier version of this work (arXiv v2). Weights for the updated model will be added alongside the code release.

Model Card

Model Description

Authors: Daniele Molino, Camillo Maria Caruso, Filippo Ruffini, Valerio Guarrasi, Paolo Soda
Model type: 3D latent diffusion (RFlow) + 3D VAE + CLIP3D text encoder for CT generation.
License: Apache 2.0 (same as code release).
Sources: Code https://github.com/cosbidev/Text2CT | Paper https://arxiv.org/abs/2506.00633
Demo: Use diff_model_demo.py from the code release for a one-off generation from text.

Intended Use

Direct use: Research/experimentation on text-conditioned 3D CT synthesis; generating synthetic data for benchmarking or augmentation.
Downstream use: Fine-tuning or integration into broader research pipelines.
Out of scope: Clinical decision-making, diagnostic use, or deployment without proper validation and approvals.

Risks & Limitations

Trained on CT-RATE; may encode dataset biases and is not validated for clinical use.
Synthetic outputs may contain artifacts; do not use for patient care.

Files included

autoencoder_epoch273.pt — 3D VAE for latent compression/decoding.
unet_rflow_200ep.pt — Diffusion UNet trained with rectified flow.
CLIP3D_Finding_Impression_30ep.pt — CLIP3D weights for encoding reports.

How to Get Started (Python)

python

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="dmolino/text2ct-weights",
    repo_type="model",
    local_dir="your_local_path",
)

Use these in the code release configs:

Config field	Points to
`trained_autoencoder_path`	autoencoder
`existing_ckpt_filepath` / `model_filename`	unet
`clip_weights`	clip

Training Data (for these weights)

CT-RATE dataset (public on Hugging Face) for CT volumes and reports.

Training Procedure (summary)

CLIP3D trained for vision-language alignment on CT + reports.
VAE checkpoint from https://github.com/Project-MONAI/tutorials/tree/main/generation/maisi.
Diffusion UNet trained with rectified flow (RFlow) in latent space, conditioned on text embeddings.

Evaluation

See paper for quantitative and qualitative results.

Further Information

1,000 generated CT scans are available at https://huggingface.co/datasets/dmolino/CT-RATE_Generated_Scans.

Environmental Impact

Not reported. Training used a multi-GPU setup.

Citation

If you use these weights or code, please cite:

bibtex

@misc{molino2026alignment,
      title={From Alignment to Synthesis Contrastive Volumetric Grounding for Text-to-CT Generation}, 
      author={Daniele Molino and Camillo Maria Caruso and Filippo Ruffini and Paolo Soda and Valerio Guarrasi},
      year={2026},
      eprint={2506.00633},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2506.00633}, 
}