README for Pathway Vision Transformer
PaViT is a Pathway Vision Transformer (PaViT)-based image recognition model developed by Ajibola Emmanuel Oluwaseun. The model is inspired by Google's PaLM (Pathways Language Model) and aims to demonstrate the potential of using few-shot learning techniques in image recognition tasks.
Model Performance
PaViT was trained on a 4GB RAM CPU using a dataset of 15000 Kaggle images of 15 classes, achieving a remarkable 88% accuracy with 4 self-attention heads. The model's accuracy further improved to 96% when trained with 12 self-attention heads and 12 linearly stacked linear layers. These results demonstrate the model's impressive performance and fast training speed on a CPU, despite being trained on a relatively small dataset.
The uploaded weight was trained on image dataset of 3 classes (Cat, Dog and Wild animal)
Usage
The model can be used for image recognition tasks by using the trained weights provided in the repository. The code can be modified to use custom datasets, and the model's performance can be further improved by adding more self-attention heads and linear layers.
Contribution
The author believes that PaViT has the potential to outperform existing Vision Transformer models and is eager to see it continue to evolve through the contributions of developers and other contributors.
Contributions to the project are welcome and can be made through pull requests. Developers can also report issues or suggest new features for the project.
License
This project is licensed under the MIT License.
How to use:
1#import Libraries
2!pip install huggingface_hub["tensorflow"]
3import matplotlib.pyplot as plt
4import cv2
5from huggingface_hub import from_pretrained_keras
On inference
1#load model
2model=from_pretrained_keras('Ajibola/PaViT')
3#load image
4image=cv2.imread('image_path')
5image=cv2.resize(image, (224, 224)) #224 is the default image size
6image=image.image.max() #Normalize the image to [0-1]
7prediction=model.predict(image)
8prediction=np.argmax(prediction, axis=-1) #Get Highest probability class
