You can deploy this model and start earning money today!

JaxNN/
vit_tiny_patch16_224.augreg_in21k

Name: vit_tiny_patch16_224.augreg_in21k
Availability: InStock

Views

No views yet

JaxNN conversion of the timm vit_tiny_patch16_224.augreg_in21k Vision Transformer checkpoint.

Model Details

Architecture: vit_tiny_patch16_224
Source: timm/vit_tiny_patch16_224.augreg_in21k

Model card for vit_tiny_patch16_224.augreg_in21k

A Vision Transformer (ViT) image classification model. Trained on ImageNet-21k (with additional augmentation and regularization) in JAX by paper authors, ported to PyTorch by Ross Wightman.

Model Details

Model Type: Image classification / feature backbone
Model Stats:
- Params (M): 9.7
- GMACs: 1.1
- Activations (M): 4.1
- Image size: 224 x 224
Papers:
- How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers: https://arxiv.org/abs/2106.10270
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale: https://arxiv.org/abs/2010.11929v2
Dataset: ImageNet-21k
Original: https://github.com/google-research/vision_transformer

Model Usage

Image Classification

python

from urllib.request import urlopen

import jax
from PIL import Image

import jaxnn

img = Image.open(urlopen(
    "https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg"
))

model = jaxnn.create_model("vit_tiny_patch16_224.augreg_in21k", pretrained=True)
model.eval()

data_config = jaxnn.data.resolve_model_data_config(model)
transforms = jaxnn.data.create_transform(**data_config, is_training=False)

x = jax.numpy.expand_dims(transforms(img), 0)
output = model(x, deterministic=True)

top5_probabilities, top5_class_indices = jax.lax.top_k(
    jax.nn.softmax(output, axis=-1) * 100,
    k=5,
)

Image Embeddings

python

from urllib.request import urlopen

import jax
from PIL import Image

import jaxnn

img = Image.open(urlopen(
    "https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg"
))

model = jaxnn.create_model(
    "vit_tiny_patch16_224.augreg_in21k",
    pretrained=True,
    num_classes=0,
)
model.eval()

data_config = jaxnn.data.resolve_model_data_config(model)
transforms = jaxnn.data.create_transform(**data_config, is_training=False)

x = jax.numpy.expand_dims(transforms(img), 0)
output = model(x, deterministic=True)

Citation

bibtex

@article{steiner2021augreg,
  title={How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers},
  author={Steiner, Andreas and Kolesnikov, Alexander and and Zhai, Xiaohua and Wightman, Ross and Uszkoreit, Jakob and Beyer, Lucas},
  journal={arXiv preprint arXiv:2106.10270},
  year={2021}
}

bibtex

@article{dosovitskiy2020vit,
  title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
  author={Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and  Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
  journal={ICLR},
  year={2021}
}

bibtex

@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}

JaxNN/vit_tiny_patch16_224.augreg_in21k

Model Details

Model card for vit_tiny_patch16_224.augreg_in21k

Model Details

Model Usage

Image Classification

Image Embeddings

Citation

JaxNN/
vit_tiny_patch16_224.augreg_in21k