Model Card for Model ID
This modelcard aims to be a base template for new models. It has been generated using
this raw template.
Model Details
Model Description
- Developed by: Luis Zúñiga
- Model type: Image Classification
- License: Apache 2.0
- Finetuned from model [optional]: google/vit-base-patch16-224-in21k
Model Sources [optional]
- Repository: [More Information Needed]
- Paper [optional]: [More Information Needed]
- Demo [optional]: [More Information Needed]
Uses
The main purpose of this model is to serve as a text representation tool for image classification of social media publications (mainly tweets) related to COVID-19 in Spanish.
However, this only a part of a multimodal model, where an additional text model is used to represent text and then, with a fusion method, text and images can be combined for a better representation of publications.
Direct Use
This model can be used to directly assess the sentiment polarity of images of social media publications. However, the main usage is to be combined with the
text model for multimodal sentiment analysis.
Bias, Risks, and Limitations
The dataset used to train this model contains tweets related to COVID-19 and many other subjects (like sports, politics, economics, to name a few), so it does not specializes in a particular topic.
[More Information Needed]
How to Get Started with the Model
Use the code below to get started with the model.
1model_ckpt = 'google/vit-base-patch16-224-in21k'
2img_model = ViTModel.from_pretrained('lzun/mcovmex-image')
3img_feature_extractor = AutoFeatureExtractor.from_pretrained(img_model)
4
5inputs = img_feature_extractor(images=img, return_tensors="pt")
6outputs = img_model(**inputs)
7last_hidden_states = outputs.last_hidden_state
Training Details
Training Data
The model is trained with the MCOVMEX dataset, specifically the images of tweets. The classes are positive (1), negative (-1), neutral (0) and spam (2). However, spam and neutral classes can be combined together to form a three class classification problem. The training data is not available due to its sensitive content, but can be shared upon reasonable request.
Evaluation
Metrics
As recommended, we use Matthew's Correlation Coefficient as the main evaluation metric due to the data imbalanceness. However, we keep track of accuracy, balanced accuracy and weighted F1.
Results
[More Information Needed]
Summary
Citation [optional]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Model Card Contact
[More Information Needed]
