Multiformer is a multi-task vision transformer architecture designed to provide strong perception capabilities with a nimble and lightweight architecture.
This model uses a custom branch of the transformers library, which can be installed easily using the instructions below.
For training and evaluation, a custom MultitaskTrainer class is used that can handle complex nested losses and successfully log them to wandb.
A
training/eval and
inference script are both available in the project repository.
You should now be able to run the following code to load a Multiformer-M0 with pretrained weights:
1from transformers import AutoModel
2
3multiformer = AutoModel.from_pretrained("FoamoftheSea/multiformer-m0")
