Model description
This model is a fine-tuned version of
microsoft/deberta-v3-large used in the paper [Add link].
The original pre-traind model has been further trained on a dataset of 4,156 annotated sentences from Bank of England officials to predict whether the sentences contain people-centered language or not – that is, whether they are relatable to the wider public or social groups and to people’s lived experiences.
The model picks up references to (i) the experiences of, or risks for, (particular categories of) people, (ii) the public interest, (iii) what is expected from, or known about, the behavior of (particular categories of) consumers, (iv) societal inequalities in one form or another, and (v) understandings that are based on engagement with citizens (including survey work).
The training data contains 945 (22.7%) instances of people-centered sentences.
It achieves the following results on the held-out evaluation set:
- Loss: 0.2420
- Accuracy: 0.9467
- F1 Macro: 0.8536
- Kappa Score: 0.7072
- Accuracy Balanced: 0.8621
- Precision Macro: 0.8456
- Recall Macro: 0.8621
Training and evaluation
More information on the training data and fine-tuning process can be found in the Supplementary Material of the above publication.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1.7e-05
- train_batch_size: 16
- eval_batch_size: 40
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.06
- num_epochs: 6
- mixed_precision_training: Native AMP
Framework versions
- Transformers 4.53.1
- Pytorch 2.6.0+cu124
- Datasets 2.14.4
- Tokenizers 0.21.2
