SAM 3 for Insects — checkpoint_18
Meta's SAM 3 image model, fine-tuned for detection and instance segmentation of terrestrial
arthropods and served through flat-bug-style pyramid tiling so that small specimens in large trap
images are recovered rather than lost to downsampling.
One instance mask, one bounding box and one confidence per insect, as COCO polygons. It is a
single-class detector: it finds arthropods, it does not identify them.
| |
|---|
| Code, CLI and Colab notebook | github.com/adambasha0/SAM3-for-Insects-segementation |
| Try it in Colab | notebook |
| Base model | SAM 3 image model |
| Training data | The flat-bug aggregate — 23 insect datasets |
| Fine-tuning run | flatbug_medium_ft, epoch 18 of 20, 1008 px, lr 8e-5 |
| File | checkpoint_18_inference.pt — 3.14 GB, fp32, unmodified |
| SHA-256 | dd8a6ce0402a6c2d00b2849a3e08becc6f3aa4ececdc526580a54539c9c41829 |
Usage
1git clone https://github.com/adambasha0/SAM3-for-Insects-segementation.git
2cd SAM3-for-Insects-segementation
3pip install -e ".[app]"
1from sam3_insect import InsectPredictor, annotations_to_coco, resolve_checkpoint
2
3predictor = InsectPredictor(resolve_checkpoint("hf"))
4
5result = predictor.predict("trap_photo.jpg")
6strong = [a for a in result.annotations if a["score"] >= 0.4]
7print(f"{len(strong)} insects found")
8
9predictor.render(result, score_threshold=0.4).save("overview.jpg")
10coco = annotations_to_coco(result)
Or fetch the file directly:
1from huggingface_hub import hf_hub_download
2path = hf_hub_download("tea98/sam3-for-insects-segmentation", "checkpoint_18_inference.pt")
Inference needs no HuggingFace token and no licence acceptance: this checkpoint carries every
weight the model uses, so the gated facebook/sam3 repository is never touched.
Choosing a threshold
SAM 3's decoder spends a fixed budget of 200 object queries in full on every tile and has no
per-query "nothing here" output — a query declines only by scoring low. So on a tile holding N
insects you get ~200 detections above 0.005 and roughly N above 0.5, at the same recall. That
is an operating-point property of DETR-style detectors, not a defect from fine-tuning.
In the fine-tuning domain real detections usually score above 0.8. The library runs inference at
0.02 and the demo app filters its display at 0.4. Fix a threshold before reporting a count.
Known limitations
- Single class — no taxonomy.
- Masks run tight, most visibly on hairy or translucent specimens and on legs and antennae.
Lower
MASK_THRESHOLD towards 0.3 when mask area matters.
- Very small objects still need help; raise
SCALE_BEFORE to 1.5–2.0.
- Domain shift costs precision more than recall — on unfamiliar imagery, raise the threshold.
- Dense clusters are merged or split inconsistently; NMS at IoU 0.2 will suppress a genuinely
overlapping pair.
- fp16 does not work.
text_projection holds values up to 9.58e18, past fp16's 65504 ceiling,
so casting turns those weights into inf. Keep fp32 weights; activations are autocast at runtime.
The full model card, including how the weights' fidelity was verified, is
in the repository.
Licence and citation
Derivative work of the SAM 3 materials, governed by the
SAM License — redistribution must carry
a copy of that licence and stay within Meta's Acceptable Use Policy. Cite
SAM 3 and
flat-bug alongside this work.
Fine-tuned at Philipps-Universität Marburg, 2026.
