Dip-ViT — Forensically Relevant Insect Classification
This repository contains the models, datasets, and application files for the Research Project: A Comparative Evaluation of Vision Transformer and Convolutional Neural Network Performance for Forensically Relevant Dipteran Classification
Author: Luka Kowalchuk
University: University of Amsterdam
Year: 2025–2026
Student number: 13326015
Project Overview
This work aimed to support forensic entomology investigations by accelerating insect identification, a key component in postmortem interval estimation and wildlife crime investigations. The main backbone architecture is BioCLIP, a Vision Transformer pretrained on images of the biological domain, finetuned using Low Rank Adaptation (LoRA) on a database developed by the Wildlife Forensic Academy. The research demonstrated that Vision Transformers outperformed a previously developed CNN, while also identifying important limitations in the existing insect database. Explainable AI techniques were explored to improve model interpretability, and a Streamlit-based demonstration application was developed for real-world implementation.
Repository Contents
streamlit_files/
All files required to run the Streamlit demonstration app:
BioCLIP-1_lora_best_flies.pt / BioCLIP-2_lora_best_flies.pt — best LoRA finetuned models for adult fly classification
BioCLIP-1_lora_best_maggots.pt / BioCLIP-2_lora_best_maggots.pt — best LoRA finetuned models for maggot classification
class_names.json — class label mappings for each model/mode combination
train_embeddings.npy / train_labels.npy — precomputed KNN embeddings and labels for open-set recognition
WFA-logo.webp, uvalogo_regular_p_en.jpg — application assets
datasets/
final_dataset.zip — current dataset used for training and evaluation, containing ~5000 images across forensically relevant Dipteran species (Chrysomya albiceps, C. marginalis, C. chloropyga, Lucilia sericata/cuprina, Muscidae, M. Synthesiomya nudiseta), for both adult flies and maggots
final_dataset_old_version.zip — original dataset from the preceding project, retained for reproducibility
Contact
