This model is a Japanese BERT_BASE fine-tuned on the WRIME dataset. It was trained as part of the paper "Emotion Analysis of Writers and Readers of Japanese Tweets on Vaccinations". Fine-tuning code is available at this repo.

This model can be used to predict intensities scores for eight emotions for writers and readers. Please refer to the Fine-tuning data section for the list of emotions.

Because of the regression fine-tuning task, it is possible for the model to infer scores outside of the range of the scores of the fine-tuning data (score < 0 or score > 4).

The Japanese BERT_BASE fine-tuned was cl-tohoku/bert-base-japanese-v2. Please refer to their model card for details regarding the model architecture, tokenization, pretraining data, and pretraining procedure.

The model is fine-tuned on WRIME, a dataset of Japanese Tweets annotated with writer and reader emotion intensities. We use version 1 of the dataset. Each Tweet is accompanied by a set of writer emotion intensities (from the author of the Tweet) and three sets of reader emotions (from three annotators). The emotions follow Plutchhik's emotions, namely:

These emotion intensities follow a four-point scale:

The BERT is fine-tuned to directly regress the emotion intensities of the writer and the averaged emotions of the readers from each Tweet, meaning there are 16 outputs (8 emotions per writer/reader).

The fine-tuning was inspired by common BERT fine-tuning procedures. The BERT was fine-tuned on WRIME for 3 epochs using the AdamW optimizer with a learning rate of 2e-5, β₁=0.9, β₂=0.999, weight decay of 0.01, linear decay, a warmup ratio of 0.01, and a batch size of 32. Training was conducted with an NVIDIA Tesla K80 and finished in 3 hours.

Below are the MSEs of the BERT on the test split of WRIME.