Dia-LLaMA fine-tuned on RadGenome-ChestCT
Full fine-tune of a Dia-LLaMA-architecture 3D CT report generator
(vision encoder + perceiver + diagnosis-guidance module + LLM decoder),
trained on the RadGenome-ChestCT benchmark's train split (22,942 volumes),
evaluated on its held-out test split (1,564 volumes).
Built with Llama — base language model is meta-llama/Meta-Llama-3.1-8B-Instruct,
used and redistributed here in fine-tuned form under the Llama 3.1
Community License.
Deviations from the original Dia-LLaMA paper (all disclosed)
- Dataset: RadGenome-ChestCT, not the paper's CTRG-Chest-548K.
- Backbone:
meta-llama/Meta-Llama-3.1-8B-Instruct, not the paper's
Llama-2-7b-chat-hf.
- Label set: this project's pinned 18 CT-RATE abnormality categories,
not Dia-LLaMA's native 14 CheXbert-style chest-X-ray conditions.
- Epoch count: capped below the paper's own
train.sh default,
for single-GPU/single-run feasibility at this dataset's scale.
- Diagnosis-guidance labels: derived via an offline RadBERT/CT-RATE
clinical labeler over ground-truth report text (same offline-labeling
approach as the original authors, different labeler/label-set).
- Training driver: a project-owned
finetune.py on stock
transformers.Trainer, not the authors' own train.py fork.
Salvaged from public pretrained sources: ViT3D vision encoder + perceiver
(Reg2RG's released RadFM-derived checkpoints). Trained from scratch: the
fc projection, LoRA adapter (r=8/alpha=32/dropout=0.1), and the
diagnosis-guidance module (dimensionally/semantically incompatible with
the paper's released 14-category/Llama-2 version).
Evaluation (RadGenome-ChestCT test split, n=1564)
Scored with this project's shared Clinical Efficacy + NLG metric suite
(src/metrics/report.py::MetricsReport) — the same pipeline used for
every other baseline in this comparison.
| Metric | Value |
|---|
| Clinical Efficacy F1 (macro) | 0.2510 |
| Clinical Efficacy F1 (micro) | 0.2899 |
| Clinical Efficacy precision (macro) | 0.2848 |
| Clinical Efficacy recall (macro) | 0.2989 |
| BLEU-1 | 0.5093 |
| BLEU-4 | 0.2403 |
| ROUGE-L | 0.2827 |
| METEOR | 0.4228 |
| CIDEr-D | 0.0400 |
| BERTScore F1 | 0.8767 |
Files
pytorch_model.bin — full fine-tuned model state dict (vision encoder +
perceiver + fc + LoRA-adapted Llama-3.1-8B-Instruct + diagnosis-guidance
module), step 28670 (final).trainer_state.json — HF Trainer training-history log (loss curve,
step/epoch record).
Optimizer/scheduler/RNG-state checkpoint-resume artifacts are intentionally
not included (training-resume-only, not needed to use or reproduce the
model's outputs).
