Face Detection using DEtection TRansformers from Facebook AI 🚀
PyTorch 1.5 +
torch vision 0.6 +
This repository includes
Training Pipeline for DETR on Custom dataset
Wider Face Dataset annotaions and images
Evaluation on test dataset
Trained weights for Wider Face Dataset in release page
Metrics Visualization
About Model
DETR or DEtection TRansformer is Facebook’s newest addition to the market of available deep learning-based object detection solutions. Very simply, it utilizes the transformer architecture to generate predictions of objects and their position in an image. DETR is a joint Convolutional Neural Network (CNN) and Transformer with a feed-forward network as a head. This architecture allows the network to reliably reason about object relations in the image using the powerful multi-head attention mechanism inherent in the Transformer architecture using features extracted by the CNN.
DETR Architecutre
Face Dataset
Dataset Image
I've used WIDER FACE dataset which is a publicly available face detection benchmark dataset, consisting of 32,203 images and label 393,703 faces with a high degree of variability in scale, pose and occlusion as depicted in the sample images. WIDER FACE dataset is organized based on 61 event classes. For each event class, the original dataset was split into 40%/10%/50% as training, validation and testing sets.
By compiling the give code, the dataset will be automatically downloaded but you can download it manually from the official website or from my github release page.
In dataloader/face.py, I set the maximum width of images in the random transform to 800 pixels. This should allow for training on most GPUs, but it is advisable to change back to the original 1333 if your GPU can handle it.
Model
We're going to use DETR with a backbone of Resnet 50, pretrained on COCO 2017 dataset. AP is computed on COCO 2017 val5k, and inference time is over the first 100 val5k COCO images, with torchscript transformer. If you want to use other DETR models, you can find them in model zoo below.
Run all the cells of detr_custom_dataset.ipynb to train your model without any errors in Google Colaboratory.
Follow this readme to understand the training pipeline of DETR and evaluation on test images.
Results
COCO Evaluation Metrics on Validation Dataset (After 15 epochs of training)
It took me 4:59:45 hours to finish 15 epochs with batch_size=16 using Tesla P100-PCIE. If you want better accuracy, you can train more epochs.
bash
1IoU metric: bbox
2 Average Precision (AP) @[IoU=0.50:0.95 |area= all |maxDets=100]=0.3933 Average Precision (AP) @[IoU=0.50|area= all |maxDets=100]=0.7664 Average Precision (AP) @[IoU=0.75|area= all |maxDets=100]=0.3705 Average Precision (AP) @[IoU=0.50:0.95 |area= small |maxDets=100]=0.0556 Average Precision (AP) @[IoU=0.50:0.95 |area=medium |maxDets=100]=0.3917 Average Precision (AP) @[IoU=0.50:0.95 |area= large |maxDets=100]=0.6158 Average Recall (AR) @[IoU=0.50:0.95 |area= all |maxDets=1]=0.2019 Average Recall (AR) @[IoU=0.50:0.95 |area= all |maxDets=10]=0.44810 Average Recall (AR) @[IoU=0.50:0.95 |area= all |maxDets=100]=0.50011 Average Recall (AR) @[IoU=0.50:0.95 |area= small |maxDets=100]=0.19412 Average Recall (AR) @[IoU=0.50:0.95 |area=medium |maxDets=100]=0.51913 Average Recall (AR) @[IoU=0.50:0.95 |area= large |maxDets=100]=0.706
