This is a MAE trained on Anime dataset. The main goal is to have a model efficient for image search, retrival and clustering.
There are 2 parts of this model, the encoder and decoder. The encoder encode the full images into 8x512 embedding and the masked out image into 8 (28x28/10) x 512 embedding. The decoder try to reconstruct that image.
Model arch is LocalViT small but with 16 layers instead of 12, Decoder is a simple transformers model with LocalViT style MLP.
