The following are the differences from the original implementation:
Dropout ratio is set to 0.2(The original was 0.8, but this value showed no loss reduction at all)
Automatic Mixed Precision was used.
The code used for the training is shown in the train.py of this repository.
The training took about 30 hours with single RTX4090.
Loss curve(loss: 0.0400 - accuracy: 0.9894 at the 90 epoch):
loss_curve_dropout02
The data(FractalDB-1k (1k categories x 1k instances; Total 1M images). [Dataset (13GB)]) used for training, downloaded from here:.
Reference:
@article{KataokaIJCV2022,
author={Kataoka, Hirokatsu and Okayasu, Kazushige and Matsumoto, Asato and Yamagata, Eisuke and Yamada, Ryosuke and Inoue, Nakamasa and Nakamura, Akio and Satoh, Yutaka},
title={Pre-training without Natural Images},
article={International Journal on Computer Vision (IJCV)},
year={2022},
}
@inproceedings{KataokaACCV2020,
author={Kataoka, Hirokatsu and Okayasu, Kazushige and Matsumoto, Asato and Yamagata, Eisuke and Yamada, Ryosuke and Inoue, Nakamasa and Nakamura, Akio and Satoh, Yutaka},
title={Pre-training without Natural Images},
booktitle={Asian Conference on Computer Vision (ACCV)},
year={2020},
}
