LADI-v2-classifier-large is based on microsoft/swinv2-large-patch4-window12to16-192to256-22kto1k-ft and fine-tuned on the MITLL/LADI-v2-dataset. LADI-v2-classifier is trained to identify labels of interest to disaster response managers from aerial images.

📘 NOTE 📘 This model is the main version of the large model which is trained on all splits of the LADI v2 dataset. It is intended for deployment and fine-tuning purposes. If you are interested in reproducing the results of our paper, see the 'reference' versions of the classifiers MITLL/LADI-v2-classifier-small-reference and MITLL/LADI-v2-classifier-large-reference models, which are trained only on the training split of the dataset.

Model Details

Model Description

The model architecture is based on swinv2 and fine-tuned on the LADI v2 dataset, which contains 10,000 post-disaster aerial images from 2015-2023 labeled by volunteers from the Civil Air Patrol. The images are labeled using multi-label classification for the following classes:

bridges_any
buildings_any
buildings_affected_or_greater
buildings_minor_or_greater
debris_any
flooding_any
flooding_structures
roads_any
roads_damage
trees_any
trees_damage
water_any

How to Get Started with the Model

LADI-v2-classifier-large is trained to identify features of interest to disaster response managers from aerial images. Use the code below to get started with the model.

The simplest way to perform inference is using the pipeline interface

python

from transformers import pipeline
image_url = "https://fema-cap-imagery.s3.amazonaws.com/Images/CAP_-_Flooding_Spring_2023/Source/IAWG_23-B-5061/A0005/D75_0793_DxO_PL6_P.jpg"

pipe = pipeline(model="MITLL/LADI-v2-classifier-large")
print(pipe(image_url))

[{'label': 'buildings_any', 'score': 0.9994631409645081},
 {'label': 'bridges_any', 'score': 0.9981274008750916},
 {'label': 'flooding_structures', 'score': 0.9974740147590637},
 {'label': 'roads_any', 'score': 0.9966784715652466},
 {'label': 'water_any', 'score': 0.9921613335609436},]

For finer-grained control, see below:

python

from transformers import AutoImageProcessor, AutoModelForImageClassification
import torch
import requests
from PIL import Image
from io import BytesIO

image_url = "https://fema-cap-imagery.s3.amazonaws.com/Images/CAP_-_Flooding_Spring_2023/Source/IAWG_23-B-5061/A0005/D75_0793_DxO_PL6_P.jpg"

img_data = requests.get(image_url).content
img = Image.open(BytesIO(img_data))

processor = AutoImageProcessor.from_pretrained("MITLL/LADI-v2-classifier-large")
model = AutoModelForImageClassification.from_pretrained("MITLL/LADI-v2-classifier-large")

inputs = processor(img, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predictions = torch.sigmoid(logits).detach().numpy()[0]
labels = [(model.config.id2label[idx], predictions[idx]) for idx in range(len(predictions))]
print(labels)

[('bridges_any', 0.9981274008750916),
 ('buildings_any', 0.9994631409645081),
 ('buildings_affected_or_greater', 0.9810144901275635),
 ('buildings_minor_or_greater', 0.010010059922933578),
 ('debris_any', 0.053571492433547974),
 ('flooding_any', 0.9028007388114929),
 ('flooding_structures', 0.9974740147590637),
 ('roads_any', 0.9966784715652466),
 ('roads_damage', 0.6155700087547302),
 ('trees_any', 0.9893258810043335),
 ('trees_damage', 0.49921801686286926),
 ('water_any', 0.9921613335609436)]

Citation

BibTeX:

@misc{ladi_v2,
      title={LADI v2: Multi-label Dataset and Classifiers for Low-Altitude Disaster Imagery}, 
      author={Samuel Scheele and Katherine Picchione and Jeffrey Liu},
      year={2024},
      eprint={2406.02780},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Developed by: Jeff Liu, Sam Scheele
Funded by: Department of the Air Force under Air Force Contract No. FA8702-15-D-0001
License: MIT
Finetuned from model: microsoft/swinv2-large-patch4-window12to16-192to256-22kto1k-ft

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

This material is based upon work supported by the Department of the Air Force under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Department of the Air Force.

The software/firmware is provided to you on an As-Is basis

Delivered to the U.S. Government with Unlimited Rights, as defined in DFARS Part 252.227-7013 or 7014 (Feb 2014). Notwithstanding any copyright notice, U.S. Government rights in this work are defined by DFARS 252.227-7013 or DFARS 252.227-7014 as detailed above. Use of this work other than as specifically authorized by the U.S. Government may violate any copyrights that exist in this work.