You can deploy this model and start earning money today!

shantam00/
CWCD

medical-report-generation

Views

No views yet

🩻 [MIDL 2026 Oral] CWCD Category-Specific LoRA Adapters

Shantam Srivastava, Mahesh Bhosale, David Doermann, Mingchen Gao

University at Buffalo, The State University of New York

Paper

Oral Presentation

Code

CWCD was accepted as a full paper at Medical Imaging with Deep Learning (MIDL) 2026 and selected for an oral presentation (oral selection rate below 8%).

📖 Model Description

This repository contains the eight category-specific rank-1 LoRA adapters released with CWCD: Category-Wise Contrastive Decoding for Structured Medical Report Generation. CWCD generates a structured findings report through independent forward passes under eight anatomical categories.

At inference time, each adapter is used for both the normal X-ray and its corresponding category-masked X-ray. CWCD contrasts the resulting base and masked log-probabilities, applies Plausibility-Based Vocabulary Subselection, and greedily selects the next token. The eight category-wise findings are combined into the final structured findings report.

The complete inference implementation and usage instructions are available in the CWCD GitHub repository.

🤗 Available Adapters

Anatomical category	Adapter directory	LoRA rank	Target modules
Lungs and Airways	`adapters/lungs`	1	`q_proj`, `v_proj`
Musculoskeletal and Chest Wall	`adapters/wall`	1	`q_proj`, `v_proj`
Cardiovascular	`adapters/cardiovascular`	1	`q_proj`, `v_proj`
Pleura	`adapters/pleura`	1	`q_proj`, `v_proj`
Hila and Mediastinum	`adapters/hila`	1	`q_proj`, `v_proj`
Tubes, Catheters, and Support Devices	`adapters/tubes`	1	`q_proj`, `v_proj`
Abdominal	`adapters/abdominal`	1	`q_proj`, `v_proj`
Other	`adapters/other`	1	`q_proj`, `v_proj`

Each directory contains:

text

adapter_config.json
adapter_model.bin

🧠 CWCD Inference

For each anatomical category, the GitHub inference pipeline:

loads the corresponding category-specific adapter;
obtains the base distribution from the normal X-ray;
obtains the masked distribution after blacking out that category's bounding boxes;
contrasts the two distributions in log-probability space;
applies Plausibility-Based Vocabulary Subselection;
generates the category-wise findings and combines all eight sections.

The contrastive score is:

score(token) = (1 + alpha) * log P_base(token) - alpha * log P_masked(token)

The released defaults are alpha=1.0 and beta=0.5.

🚀 Using the Adapters

The adapters are downloaded automatically by inference.py through huggingface_hub.snapshot_download(). Only the adapter configuration and weight files are retrieved.

bash

git clone https://github.com/shantamsrivastava/CWCD.git
cd CWCD

python -m pip install -r requirements.txt
python -m pip install git+https://github.com/microsoft/LLaVA-Rad.git

python inference.py \
  --image chest_xray.png \
  --boxes boxes.example.json \
  --alpha 1.0 \
  --beta 0.5 \
  --output report.json

See the GitHub README for environment setup, input format, standard category-wise inference, prompts, and all CLI options.

📦 Repository Structure

text

adapters/
├── abdominal/
├── cardiovascular/
├── hila/
├── lungs/
├── other/
├── pleura/
├── tubes/
└── wall/

⚠️ Intended Use and Limitations

These adapters are intended strictly for research and educational use. CWCD has not been approved or validated for clinical or diagnostic deployment and must not be used to make medical decisions or inform patient care.

Generated findings can contain omissions, incorrect statements, or hallucinations. Performance depends on the base model, input image quality, category-specific bounding boxes, and the domain represented by the evaluation data. Users are responsible for protecting patient privacy and following all applicable data-use agreements.

🤝 Acknowledgements

CWCD builds on LLaVA-Rad, LLaVA, Vicuna, and PEFT.

📑 Citation

bibtex

@inproceedings{srivastava2026cwcd,
  title={CWCD: Category-Wise Contrastive Decoding for Structured Medical Report Generation},
  author={Srivastava, Shantam and Bhosale, Mahesh and Doermann, David and Gao, Mingchen},
  booktitle={Medical Imaging with Deep Learning},
  series={Proceedings of Machine Learning Research},
  volume={315},
  pages={868--893},
  year={2026},
  publisher={PMLR}
}

shantam00/CWCD