This is a fine-tuned version of the GPT-2 model for sentiment analysis on tweets. The model has been trained on the mteb/tweet_sentiment_extraction dataset to classify tweets into three sentiment categories: Positive, Neutral, and Negative. It uses the Hugging Face Transformers library and achieves an evaluation accuracy of 76%.

Model Details

Developer/Owner

Creator: Ashish Y Beary
Contact: ashishbearyy1@gmail.com

Model Type

Architecture: GPT-2
Fine-Tuned Task: Sentiment Analysis

Dataset

Name: mteb/tweet_sentiment_extraction
Description: A dataset for extracting and classifying sentiment in tweets.
Language: English
Size: 1,000 samples used for training and 1,000 for evaluation.

Training Configuration

Tokenizer: GPT-2 Tokenizer (with EOS token as pad token)
Optimizer: AdamW
Learning Rate: 1e-5
Epochs: 3
Batch Size: 1
Hardware Used: A100

Performance

Accuracy: 76%
Evaluation Metric: Accuracy
Validation Split: 10% of the dataset.

Usage

This model is designed for sentiment analysis of tweets or other short social media text. Given an input text, it predicts the sentiment as Positive, Neutral, or Negative.

Example Code

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load the model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("charlie1898/gpt2_finetuned_twitter_sentiment_analysis")
model = AutoModelForSequenceClassification.from_pretrained("charlie1898/gpt2_finetuned_twitter_sentiment_analysis")

# Example input
text = "I love using Hugging Face models!"
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
predicted_class = torch.argmax(outputs.logits).item()
print(f"Predicted sentiment class: {predicted_class}")

# Limitations
- ** Bias **: The dataset may contain biased or harmful text, potentially influencing predictions.
- ** Domain Limitations **: Optimized for English tweets; performance may degrade on other text types or languages.

# Ethical Considerations
This model should be used responsibly. Be aware of biases in the training data and avoid deploying the model in sensitive or high-stakes applications without further validation.

# Acknowledgments
- Hugging Face Transformers library
- mteb/tweet_sentiment_extraction dataset