mEye: A Deep Learning Tool for Pupillometry

⭐ MEYE is available on MATLAB! Check it out here

Check out pupillometry.it for a ready-to-use web-based mEye pupillometry tool!

Requirements

• tensorflow >= 2.4
• imageio, imageio-ffmpeg
• scipy
• tqdm

• adabelief_tf >= 0.2.1
• pandas
• sklearn

Make Predictions with Pretrained Models

1. Download the pretrained model. If you want to use the old model, check out version v0.1 of this branch. See available models in Releases.
2. Check out the pupillometry-offline-videos.ipynb notebook for a complete example of pupillometry data analysis.
3. In alternative, we provide also the predict.py script that implements the basic loop to make predictions on video streams. E.g.:
   • bash

     1# input: webcam (default) 2# prediction roi: biggest central square crop (default) 3# outputs: predictions.mp4, predictions.csv (default) 4predict.py path/to/model
   • bash

     1# input: video file 2# prediction roi: left=80, top=80, right=208, bottom=208 3# outputs: video_with_predictions.mp4, pupil_metrics.csv 4predict.py path/to/model path/to/video.mp4 -rl 80 -rt 80 -rr 208 -rb 208 -ov video_with_predictions.mp4 -oc pupil_metrics.csv
   • bash

     1# check all parameters with 2predict.py -h

Training Models

1. Download our dataset (NN_human_mouse_eyes.zip, 246.4 MB) or prepare your dataset following our dataset's structure.

   If you need to annotate your dataset, check out pLabeler, a MATLAB software for labeling pupil images.

   The dataset should be placed in data/<dataset_name>.
2. If you are using a custom dataset, edit train.py to perform the train/validation/test split of your data.
3. Train with default parameters:
   python train.py -d data/<dataset_name>

• For a list of available parameters, run
  python train.py -h

MATLAB support

References

Dataset

 @dataset{raffaele_mazziotti_2021_4488164,
   author       = {Raffaele Mazziotti and Fabio Carrara and Aurelia Viglione and Lupori Leonardo and Lo Verde Luca and Benedetto Alessandro and Ricci Giulia and Sagona Giulia and Amato Giuseppe and Pizzorusso Tommaso},
   title        = {{Human and Mouse Eyes for Pupil Semantic Segmentation}},
   month        = feb,
   year         = 2021,
   publisher    = {Zenodo},
   version      = {1.0},
   doi          = {10.5281/zenodo.4488164},
   url          = {https://doi.org/10.5281/zenodo.4488164}
 }