Instance segmentation of 7 types of physical damage on cars, for insurance, rental and resale inspection workflows. Given a photo, the model outputs a pixel mask, a bounding box, a damage class and a confidence score for every damaged region it finds.

Examples

Five damaged regions across three classes

Tear and paint scratches

Side panel

Single crease, high confidence

A failure case — read this one. The dent on this hood is obvious to a human, but at the default threshold of 0.7 the model returns nothing. It only appears at 0.5, scoring 0.57:

The demo images do not cover all 7 classes — the only available example of Vỡ kính / Broken glass had a readable licence plate and a phone number burned into the frame, so it was excluded rather than published.

Installation

pip install cardamage

Quick start

python

import torch
from PIL import Image
from cardamage import AutoModel

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = AutoModel.from_pretrained("Naiscorp/car-damage-maskrcnn-r101-dc5").to(device).eval()

overlay = model.inference(Image.open("car.jpg"))   # PIL.Image with masks drawn
overlay.save("damage.png")

Structured output instead of a picture:

python

r = model.predict(Image.open("car.jpg"))
r["boxes"]      # (N, 4) float32, xyxy in ORIGINAL image coordinates
r["scores"]     # (N,)   float32
r["classes"]    # (N,)   int64, 0..6
r["labels"]     # list[str], Vietnamese
r["labels_en"]  # list[str], English
r["masks"]      # (N, H, W) bool, pasted back to the original resolution

Labels

id	Tiếng Việt	English
0	Móp lõm	Dent
1	Trầy sơn	Paint scratch
2	Rách	Tear
3	Mất bộ phận	Missing part
4	Thủng	Puncture
5	Bể đèn	Broken lamp
6	Vỡ kính	Broken glass

Architecture


Meta architecture	Mask R-CNN (`GeneralizedRCNN`)
Backbone	ResNet-101, Dilated-C5 — `res5` dilation 2, single feature map, stride 16, 2048 ch
Normalisation	FrozenBatchNorm (all stages)
Region proposals	`StandardRPNHead`, 15 anchors/location (sizes 32–512, ratios 0.5/1/2)
ROI heads	`StandardROIHeads`, ROIAlignV2 7×7 → 2 × FC-1024
Mask head	ROIAlignV2 14×14 → 4 × conv-256 → deconv → 1×1 conv, 28×28 output
Parameters	190,900,534 (191,111,222 tensor entries including FrozenBN buffers)
Backbone init	ImageNet-pretrained `MSRA/R-101`; stem and `res2` frozen during training

Training schedule


Batch size	16
Scheduled iterations	270,000
Iterations in this checkpoint	59,999 — about 22% of the schedule
LR schedule	`WarmupMultiStepLR`: 1,000-iter linear warmup from 2e-5 to `BASE_LR` 0.02, then ×0.1 at 210,000 and 250,000

This is an intermediate checkpoint, not the end of the planned schedule. It is the one that has been running in production, which is why it is the one released.

Inference defaults (all overridable, all recorded in config.json):


Input format	BGR, shortest edge 800 px, longest edge capped at 1333 px
Pixel mean / std	`[103.53, 116.28, 123.675]` / `[1.0, 1.0, 1.0]`
Score threshold	0.7
NMS threshold	0.5
Max detections	100 per image
Mask binarisation	0.5

Training data

Internal dataset, collected in Vietnam, annotated with VGG Image Annotator and converted to COCO instance-segmentation format. The dataset itself is not published. The statistics below describe the annotated split the model was trained and evaluated on.

	Train	Test
Images	2,085	417
Annotated instances	4,715	996

Per-class instance counts:

Class	Train	Test
Trầy sơn / Paint scratch	1,461	323
Móp lõm / Dent	1,087	235
Vỡ kính / Broken glass	692	127
Rách / Tear	679	140
Mất bộ phận / Missing part	472	90
Bể đèn / Broken lamp	185	42
Thủng / Puncture	139	39

More distribution charts

Benchmarks

Accuracy

Measured on the 417-image test split described above, taken from the internal project report Car Damage Analysis (HCMC, June 2025):

Model	AP50 ↑	AP50-95 ↑	AR ↑
`mask_rcnn_R_101_DC5_3x` — this model	19.64	11.09	18.8
`mask_rcnn_R_101_FPN_3x` — a later retrain, not released here	21.97	11.92	18.7

Latency

Measured directly on this checkpoint, 1280×720 input (resized to 1333×750), full pipeline including preprocessing and mask pasting:

Device	Median	Peak VRAM
NVIDIA RTX 4090 (fp32)	81 ms/image	1.17 GB

License and attribution

model.py re-implements the inference algorithms of Detectron2 (Copyright 2019-present, Facebook, Inc. — licensed under the Apache License, Version 2.0), keeping the original module and parameter names so that checkpoints trained with Detectron2 load without any key remapping.