FocalNet model trained on ImageNet-1k at resolution 384x384. It was introduced in the paper Focal Modulation Networks by Yang et al. and first released in this repository.

Disclaimer: The team releasing FocalNet did not write a model card for this model so this model card has been written by the Hugging Face team.

Focul Modulation Networks are an alternative to Vision Transformers, where self-attention (SA) is completely replaced by a focal modulation mechanism for modeling token interactions in vision. Focal modulation comprises three components: (i) hierarchical contextualization, implemented using a stack of depth-wise convolutional layers, to encode visual contexts from short to long ranges, (ii) gated aggregation to selectively gather contexts for each query token based on its content, and (iii) element-wise modulation or affine transformation to inject the aggregated context into the query. Extensive experiments show FocalNets outperform the state-of-the-art SA counterparts (e.g., Vision Transformers, Swin and Focal Transformers) with similar computational costs on the tasks of image classification, object detection, and segmentation.

You can use the raw model for image classification. See the model hub to look for fine-tuned versions on a task that interests you.

Here is how to use this model to classify an image of the COCO 2017 dataset into one of the 1,000 ImageNet classes:

For more code examples, we refer to the documentation.