AiManatee/

RoBERTa_poem_sentiment

This model is a fine-tuned version of the FacebookAI/roberta-base transformer for the task of poem sentiment analysis. It predicts the sentiment of a given poem verse into one of four categories: negative, positive, no impact, or mixed (positive and negative).

RoBERTa_poem_sentiment was trained on the poem_sentiment dataset which consists of poem verses across four sentiment labels: negative, positive, no impact, and mixed sentiment. However, the Validation and Test subsets of the original dataset lack 'mixed' sentiment examples. To address this and ensure a thorough evaluation, data augmentation was performed: 32 'mixed' sentiment verses from different English poems were added to the Validation (16) and Test (16) subsets; the original Train subset remained intact. All the augmented samples were tested for semantic consistency, diversity (cosine similarity), length variation and novelty (ensuring the augmented data introduced new, relevant vocabulary). This strategy allowed for a more comprehensive evaluation of the model's generalization ability across all trained labels. The final model was tested on both the original dataset and the augmented dataset.

  learning_rate: 2e-5,
  weight_decay: 0.01,
  batch_size: 16,
  num_epochs: 8,
  optimizer: AdamW: betas=(0.9, 0.999), eps=1e-08
  seed: 16
  early_stopper: min_delta=0.001, patience=3

  scheduler = ReduceLROnPlateau(
    optimizer,
    mode="min",
    factor=0.5,
    patience=0,
    threshold=0.001,
    eps=1e-8,
  )

Here is how to predict the sentiment of a poem verse using this model:

{Loss: 0.5726433790155819
Accuracy: 0.8857142857142857
Precision: 0.9201298701298701
Recall: 0.8857142857142857
F1: 0.9025108225108224
}

{Loss: 0.5837492472492158
Accuracy: 0.8842975206611571
Precision: 0.8810538160090016
Recall: 0.8842975206611571
F1: 0.8813606847697756
}