Baseline model for audio classification of orthopera and hemiptera
This baseline model, utilized in the
ECOSoundSet paper, was trained to tag audio files with one or more of 86 species from the Orthoptera and Hemiptera insect orders.
Installation
To use the model, you have to install autrainer, e.g. via pip:
pip install autrainer
For more information about autrainer, please refer to:
https://autrainer.github.io/autrainer/index.htmlUsage
The model can be applied on all wav files present in a folder (<data-root>) and stored in another folder (<output-root>):
autrainer inference hf:AlexanderGbd/insects-base-cnn10-96k-t -r <data-root> <output-root>
For possible inference settings (e.g. using sliding window) and all usable parameters, please have a look at the
autrainer documentation.
The model requires an input file sampled at 96 kHz.
The "-sr 96000" flag will re-sample the audio if its sampling frequency differs from 96 kHz.
The following model usage is recommended:
autrainer inference hf:AlexanderGbd/insects-base-cnn10-96k-t /path/to/input/data /path/to/store/results -r -w 4 -s 4 -sr 96000
Here:
- r: recursive flag (recursively go through all subfolders as well).
- w: window size (here 4 seconds), i.e. length of the audio segment analyzed (in seconds).
- s: step size (here 4 seconds), i.e. how far the window moves forward after each analysis (in seconds).
- sr: target sampling rate (in Hz). If audio files don’t match this rate, they’re resampled.
- e: file extension (default is wav), i.e. the file format to process.
Training
Pretraining
The model has been originally trained on AudioSet by Kong et. al. (
https://ieeexplore.ieee.org/abstract/document/9229505).
Dataset
The model has been further trained (finetuned) on 4s long audio segments of the first version of the ECOSoundSet dataset.
Features
The audio recordings were resampled to 96kHz, as we wanted to avoid losing too much frequency information from the species. Log-Mel spectrograms were then extracted using torchlibrosa.
Training process
The model has been trained for 30 epochs. At the end of each epoch, the model was evaluated on our validation set. We release the state that achieved the best performance on this validation set. All training hyperparameters can be found inside conf/config.yaml inside the model folder. The train, dev, and test set can be accessed in the splits-folder.
Evaluation
The performance on the test set reached a (macro) f1-score of 0.568.
Acknowledgments
Please acknowledge the work which produced the original model and the ECOSoundSet dataset. We would also appreciate an acknowledgment to autrainer.
