garment-detector-seg
Instance segmentation of garments, accessories, and garment components in
photographs. RF-DETR-Seg (Apache 2.0) fine-tuned on
Fashionpedia (CC BY 4.0), predicting
all
46 Fashionpedia classes: 13 main garments (shirt/blouse … cape),
14 accessories, and 19 garment parts (collar, lapel, sleeve, pocket, neckline,
zipper, buckle, and trim classes such as rivet, ruffle, sequin).
Built for wholesale apparel manufacturing QC, not e-commerce tagging: the
segmentation masks feed deterministic color measurement, and the part classes
map to BOM trim lines and construction details in a tech pack. It is one of
two models in a pipeline (see garment-attributes for fine-grained
construction attributes).
Intended use
- Locate garments and their visible components in sample/production photos.
- Provide masks for downstream color QC (ΔE2000 against a spec target).
- Verify component presence/count against a tech pack (e.g. "2 flap pockets,
1 center-front zipper, no hood").
Out of scope: fiber content, GSM, measurements, stitch class, interior
construction — none of these are visually determinable and this model does
not pretend to output them.
Training
- Base: RF-DETR-Seg-Small (Apache 2.0), DINOv2 backbone, 33.6M params.
- Data: Fashionpedia train2020 (45,623 images, 333,401 instances), converted
by
scripts/prepare_detector_data.py (attributes stripped; masks kept).
- Recipe:
scripts/train_detector.py with configs/detector_mac.yaml
(8 epochs, effective batch 16, lr 1e-4, fp16 AMP, trained on Apple-silicon
MPS in ~37 hours).
Evaluation (Fashionpedia val2020, 1,158 images, epoch 8)
| Metric | Value |
|---|
| mask mAP@[.5:.95] | 0.352 |
| mask mAP@.5 | 0.502 |
| box mAP@[.5:.95] | 0.419 |
| box mAP@.5 | 0.549 |
| box mAP, main garments (ids 0–12) | 0.589 |
| box mAP, accessories (ids 13–26) | 0.534 |
| box mAP, garment parts (ids 27–45) | 0.204 |
Strongest classes: dress (0.87), pants (0.84), jacket and coat (approx. 0.75).
Weakest: small trims — zipper (0.10), beads/appliqué (0.05) — and rare
categories (cape, jumpsuit). Full per-class table ships in this repo.
Small trim classes (rivet, bead, sequin) score materially lower than garment
classes in all published Fashionpedia baselines; expect the same here and
check the per-class table in runs/detector before relying on a trim class
for QC decisions.
Known limitations & biases
- Fashionpedia images are street/celebrity photos of worn garments;
performance on flat-lay or on-hanger factory photos is lower. Fine-tune on
in-domain photos for production deployment (the training script supports
any COCO-format dataset).
- Occluded components (e.g. back pockets in a front photo) are not detected —
the downstream validator reports missing components as REVIEW, not FAIL,
for this reason.
- Small trims (<32 px) are frequently missed at 1024-px inference resolution.
- Dataset skews toward Western womenswear street fashion; expect weaker
performance on technical/workwear categories.
License & attribution
Weights: Apache 2.0. Training data: Fashionpedia, CC BY 4.0 — cite
Jia et al., Fashionpedia: Ontology, Segmentation, and an Attribute
Localization Dataset, ECCV 2020. Architecture: RF-DETR (Roboflow, Apache
2.0), ICLR 2026.
Usage
1from rfdetr import RFDETRSegSmall
2model = RFDETRSegSmall(pretrain_weights="checkpoint_best_ema.pth")
3detections = model.predict("sample.jpg", threshold=0.5)
Full pipeline (attributes + color + tech pack validation): see the
garment-classifier repository this model ships with.
⚠ This card predates our model-card standard
It does not report: a baseline comparison, a measured failure rate, or what it refuses.
Every model we publish should state (1) the cheap baseline and the margin over it, (2) at least
one named failure mode with a measured rate, (3) what the model refuses and why, and (4) label
provenance. This card was written before that rule and has not been retrofitted, because the
measurements needed no longer exist. Treat its numbers with more caution than our newer cards,
and prefer a model that meets the standard where one exists.
