Butterfly & moth ID: wing classifier (BioCLIP 2.5-H + hierarchical head)
A fine-grained classifier that identifies Neotropical
butterflies and nocturnal moths
from
wing photos, down to
subspecies. It is the head used by the
Butterfly ID inference Space
and the
AI Identifier web app.
The frozen BioCLIP 2.5-H backbone is not redistributed here. Get it from
imageomics/bioclip-2.5-vith14.
This repo holds the trained head and the wing-cropper.
Coverage
The label space spans Neotropical butterflies across all major families plus, as of this
version, a substantial component of nocturnal moths — 4,946 species / 7,292
subspecies-level leaves / 883 genera / 69 tribes / 24 subfamilies / 11 families:
- Butterflies — Nymphalidae, Hesperiidae (skippers), Riodinidae (metalmarks), Lycaenidae,
Pieridae, Papilionidae, plus the nocturnal Hedylidae and the day-flying Castniidae.
- Nocturnal moths (new) — predominantly Sphingidae (hawkmoths), with Saturniidae,
Geometridae, Notodontidae, Erebidae/Arctiinae, Noctuidae and others.
Hierarchical roll-up (genus → tribe → family) is currently complete for the butterflies.
The moth genera are still being mapped into the higher-rank taxonomy, so moth predictions are
most reliable at species / genus level for now; their family marginalisation is incomplete.
Sampling is very uneven. Most butterfly data sits in the Ithomiini mimicry radiation
(Sanger/Ikiam collection), and most moth data in Sphingidae. Sparsely-represented groups
(Hedylidae, Castniidae, many Hesperiidae/Lycaenidae, and the thinner moth families) are
in-scope but low-confidence — treat confident calls outside the well-sampled systems with
caution. Even within Ithomiini, Müllerian mimicry makes subspecies look-alikes genuinely hard
to tell apart.
Files
| file | what it is |
|---|
head_hier.pt | the hierarchical CosineHead / ArcFace classifier head (feat_dim 1024). Holds leaves (7,292 taxa), rank_vocab (4,946 species / 883 genera), leaf2rank, head_state_dict. |
wing_seg.pt | YOLO26-seg wing/insect segmenter used to crop the photo to the wings before BioCLIP (masks trained with SAM 3). |
region_checklist.json | per-region (side-of-Andes / country) species & subspecies checklist, for the optional geographic prior. |
Pipeline
image → (YOLO wing-crop) → BioCLIP 2.5-H embed (frozen) → CosineHead → leaf probs → SUM-marginalise to species/genus → (optional geo prior re-rank)
Usage
The full, runnable inference code lives in the Space
(
inference.py).
Minimal sketch:
1import torch, open_clip
2ck = torch.load("head_hier.pt", map_location="cpu", weights_only=False)
3leaves = ck["leaves"] # 7292 finest-rank labels (subspecies where known)
4model, _, preprocess = open_clip.create_model_and_transforms("hf-hub:imageomics/bioclip-2.5-vith14")
5# build the CosineHead (s=30, LayerNorm) from ck["head_state_dict"] (see inference.py)
6feat = model.encode_image(preprocess(img).unsqueeze(0)).float()
7# probs = head(feat).softmax(1) -> argsort for the top taxa
Performance
Out-of-fold top-1 accuracy on Sanger/Ikiam specimens (dorsal+ventral combined, with the
side-of-Andes + Ecuador prior). These are measured on the butterfly collection, the deployment target:
| Rank | Top-1 |
|---|
| Subspecies | 84.0% |
| Species | 89.7% |
| Genus | 95.4% |
| Tribe | 96.8% |
| Subfamily | 98.9% |
| Family | 99.4% |
Reliable from genus upward (≥95%). Subspecies is the hard frontier because Müllerian
mimicry produces look-alikes across species. The backbone is currently frozen (only the
head is trained); backbone fine-tuning is the main planned lever for species/subspecies gains.
Training data
The subspecies-level taxonomic identifications were compiled from
Butterflies of America,
Sangay,
Noreste, and
Cotacachi, expanded with additional Neotropical butterfly photo
databases from across the region and, for the nocturnal moths, specialist collections such as
the Sphingidae Taxonomic Inventory and other hawkmoth / saturniid resources. Figure plates from
taxonomic PDF monographs (e.g. Cock 2018,
Hawk-moths of Trinidad) were re-extracted with
layout-aware parsing (MinerU) so each single-specimen figure carries its caption's species +
subspecies; composite/multi-specimen plates were excluded.
License
CC-BY-NC-4.0 (attribution, non-commercial). The non-commercial term reflects that the
classifier's subspecies-level labels were compiled from third-party taxonomic databases
(see Training data); please credit this work and respect those sources' terms.
Disclaimer
This is an AI suggestion, not a definitive identification. It is intended as a curation
aid to flag uncertain or mislabelled identifications.
