We can import pre-trained FasterViT models with 1 line of code. First, FasterViT can be simply installed by:
pip install fastervit
A pretrained FasterViT model with default hyper-parameters can be created as in the following:
python
1>>>from fastervit import create_model
23# Define fastervit-0 model with 224 x 224 resolution45>>> model = create_model('faster_vit_0_224',6 pretrained=True,7 model_path="/tmp/faster_vit_0.pth.tar")
model_path is used to set the directory to download the model.
We can also simply test the model by passing a dummy input image. The output is the logits:
We can also use the any-resolution FasterViT model to accommodate arbitrary image resolutions. In the following, we define an any-resolution FasterViT-0
model with input resolution of 576 x 960, window sizes of 12 and 6 in 3rd and 4th stages, carrier token size of 2 and embedding dimension of
64:
python
1>>>from fastervit import create_model
23# Define any-resolution FasterViT-0 model with 576 x 960 resolution4>>> model = create_model('faster_vit_0_any_res',5 resolution=[576,960],6 window_size=[7,7,12,6],7 ct_size=2,8 dim=64,9 pretrained=True)
Note that the above model is intiliazed from the original ImageNet pre-trained FasterViT with original resolution of 224 x 224. As a result, missing keys and mis-matches could be expected since we are addign new layers (e.g. addition of new carrier tokens, etc.)
We can simply test the model by passing a dummy input image. The output is the logits:
All models use crop_pct=0.875. Results are obtained by running inference on ImageNet-1K pretrained models without finetuning.
Name
A-Acc@1(%)
A-Acc@5(%)
R-Acc@1(%)
R-Acc@5(%)
V2-Acc@1(%)
V2-Acc@5(%)
FasterViT-0
23.9
57.6
45.9
60.4
70.9
90.0
FasterViT-1
31.2
63.3
47.5
61.9
72.6
91.0
FasterViT-2
38.2
68.9
49.6
63.4
73.7
91.6
FasterViT-3
44.2
73.0
51.9
65.6
75.0
92.2
FasterViT-4
49.0
75.4
56.0
69.6
75.7
92.7
FasterViT-5
52.7
77.6
56.9
70.0
76.0
93.0
FasterViT-6
53.7
78.4
57.1
70.1
76.1
93.0
A, R and V2 denote ImageNet-A, ImageNet-R and ImageNet-V2 respectively.
Citation
Please consider citing FasterViT if this repository is useful for your work.
@article{hatamizadeh2023fastervit,
title={FasterViT: Fast Vision Transformers with Hierarchical Attention},
author={Hatamizadeh, Ali and Heinrich, Greg and Yin, Hongxu and Tao, Andrew and Alvarez, Jose M and Kautz, Jan and Molchanov, Pavlo},
journal={arXiv preprint arXiv:2306.06189},
year={2023}
}
