You can deploy this model and start earning money today!

abdullahzunorain/
rt-detr-v2-fine-tuned-PPE-Det

Name: rt-detr-v2-fine-tuned-PPE-Det
Availability: InStock

PekingU/rtdetr_v2_r50vd

finetune

mit

Views

No views yet

RT-DETRv2 Fine-Tuned — PPE Compliance Detection

A fine-tuned RT-DETRv2 (ResNet-50vd backbone) model for real-time Personal Protective Equipment (PPE) compliance monitoring from industrial CCTV footage. Trained for PT Pertamina EP Cepu (PEPC) to detect PPE compliance across 11 classes simultaneously.

⚠️ Critical usage note: This model requires use_fast=False in the image processor (see inference example). The default fast processor in transformers ≥ 5.3 applies incorrect normalization for RT-DETRv2 and will silently produce wrong predictions.

Model Details

Property	Value
Base model	`PekingU/rtdetr_v2_r50vd` (COCO pretrained)
Backbone	ResNet-50 (r50vd)
Framework	PyTorch + HuggingFace Transformers
Input resolution	800×1333 (longest edge, aspect-ratio preserved)
Number of classes	11
Training epochs	38 (early stopped, patience = 20)
Hardware	NVIDIA Quadro P3200 — 6 GB VRAM

Classes Detected

Index	Class	Type
0	person	Anchor
1	coverall_present	PPE — compliant
2	helmet_present	PPE — compliant
3	helmet_absent	PPE — violation
4	boot_present	PPE — compliant
5	glove_absent	PPE — violation
6	coverall_absent	PPE — violation
7	eyewear_absent	PPE — violation
8	boot_absent	PPE — violation
9	eyewear_present	PPE — compliant
10	glove_present	PPE — compliant

Performance

Evaluated on 964 held-out validation images using COCO evaluation protocol (pycocotools).

Metric	Value
mAP@50	62.29%
mAP@50:95	39.7%
mAP@75	40.8%
AP — small objects	26.4%
AP — medium objects	44.7%
AP — large objects	54.1%
AR@100	55.8%

Per-class difficulty (qualitative):

Strong (>80% AP): person, helmet_absent, coverall_present, helmet_present
Moderate (50–70% AP): coverall_absent, boot_absent, glove_absent
Weak (<55% AP): boot_present, eyewear_present, eyewear_absent, glove_present

glove_present is the hardest class — only 60 training examples in a 46,200-annotation dataset (133:1 imbalance vs. person).

Dataset

Property	Value
Format	COCO JSON
Total images	3,200
Total annotations	46,200
Image resolution	1280×720 (CCTV frames)
Train split	2,236 images · 31,179 annotations
Val split	964 images · 15,021 annotations
Split strategy	Stratified 70/30, seed=42

Class imbalance:

Most common: person — 7,984 annotations
Rarest: glove_present — 60 annotations (133× imbalance)
Absent classes dominate present classes across all PPE types

Object size distribution:

Small (<1,024 px²): 38.9%
Medium (1,024–9,216 px²): 37.1%
Large (>9,216 px²): 24.0%
Median bbox area: 1,757 px² — heavy small-object presence

Training Configuration

Setting	Value
Batch size	2 (effective: 16 via gradient accumulation ×8)
Learning rate	1e-4 (heads) · 1e-5 (backbone, 0.1× factor)
LR schedule	CosineAnnealingLR + 3-epoch linear warmup
Backbone freeze	Epochs 1–10 frozen; unfrozen from epoch 11
EMA decay	0.999
Gradient clip	5.0
Early stopping	patience = 20 epochs
Mixed precision	Disabled (Pascal GPU — weak FP16)

Augmentation pipeline (9 transforms): HorizontalFlip · RandomScale (±15%) · Affine (shift ±5%, rotate ±5°) · RandomCrop · BrightnessContrast · MotionBlur · GaussNoise · CLAHE · RandomShadow → Resize(800×1333)

Class imbalance strategy:

Weighted random sampling (inverse frequency, cap = 10×)
Per-class loss weighting

Inference

Image inference

python

from transformers import AutoImageProcessor, AutoModelForObjectDetection
from huggingface_hub import hf_hub_download
import torch
from PIL import Image

# Load processor — use_fast=False is REQUIRED, do not remove
processor = AutoImageProcessor.from_pretrained(
    "PekingU/rtdetr_v2_r50vd",
    use_fast=False
)

# Load base architecture then apply fine-tuned weights
model = AutoModelForObjectDetection.from_pretrained(
    "PekingU/rtdetr_v2_r50vd",
    num_labels=11,
    ignore_mismatched_sizes=True
)
weights_path = hf_hub_download(
    repo_id="abdullahzunorain/rt-detr-v2-fine-tuned-PPE-Det",
    filename="best_model.pt"
)
model.load_state_dict(
    torch.load(weights_path, map_location="cpu", weights_only=True),
    strict=False
)
model.eval()

id2label = {
    0: "person",          1: "coverall_present", 2: "helmet_present",
    3: "helmet_absent",   4: "boot_present",     5: "glove_absent",
    6: "coverall_absent", 7: "eyewear_absent",   8: "boot_absent",
    9: "eyewear_present", 10: "glove_present"
}

image = Image.open("image.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)

results = processor.post_process_object_detection(
    outputs,
    threshold=0.3,
    target_sizes=[image.size[::-1]]
)[0]

for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
    print(f"{id2label[label.item()]}: {score:.2f} @ {[round(c,1) for c in box.tolist()]}")

Video inference (frame-by-frame)

python

import cv2
from transformers import AutoImageProcessor, AutoModelForObjectDetection
from huggingface_hub import hf_hub_download
import torch
from PIL import Image

processor = AutoImageProcessor.from_pretrained(
    "PekingU/rtdetr_v2_r50vd", use_fast=False
)
model = AutoModelForObjectDetection.from_pretrained(
    "PekingU/rtdetr_v2_r50vd",
    num_labels=11,
    ignore_mismatched_sizes=True
)
weights_path = hf_hub_download(
    repo_id="abdullahzunorain/rt-detr-v2-fine-tuned-PPE-Det",
    filename="best_model.pt"
)
model.load_state_dict(
    torch.load(weights_path, map_location="cpu", weights_only=True),
    strict=False
)
model.eval()

cap = cv2.VideoCapture("video.mp4")
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break
    image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
    inputs = processor(images=image, return_tensors="pt")
    with torch.no_grad():
        outputs = model(**inputs)
    results = processor.post_process_object_detection(
        outputs, threshold=0.3, target_sizes=[image.size[::-1]]
    )[0]
    # draw boxes on frame here
cap.release()

Limitations

glove_present recall is low — 60 training examples is insufficient; the model may miss glove compliance in many scenes
Small object performance — AP_small = 26.4%; gloves and eyewear on distant workers are frequently missed
Present/absent confusion — the dataset is skewed toward absent (violation) classes, so the model is more reliable at detecting violations than compliance
Single-site training — trained on one facility's CCTV aesthetic; may not generalize to different lighting conditions, camera angles, or worker attire styles
1,532 noisy annotations — bboxes with width or height < 10px exist in training data and introduce noise for small PPE classes
No segmentation — bbox detection only; not suitable for pixel-level PPE segmentation tasks
Not for autonomous safety decisions — model predictions require human review before any enforcement action

Intended Use

Industrial PPE compliance monitoring dashboards
CCTV-based safety auditing systems
Computer vision research on workplace safety
Prototyping real-time detection pipelines

Out of scope: Autonomous enforcement, safety-critical decisions without human oversight, or environments significantly different from industrial CCTV settings.

Repository Files

File	Description
`best_model.pt`	Best checkpoint weights (epoch 18, mAP@50 = 62.29%)
`det_Cam-1895.mp4`	Sample inference video

Citation

bibtex

@misc{zunorain2026rtdetrv2ppe,
  author = {Abdullah Zunorain},
  title  = {RT-DETRv2 Fine-Tuned PPE Detection},
  year   = {2026},
  url    = {https://huggingface.co/abdullahzunorain/rt-detr-v2-fine-tuned-PPE-Det}
}