RoBERTa-based sentence-level classification model.
Similar to alex2awesome/quote-detection__roberta-base-sentence, except it's trained on a lot more data.
Scores this accuracy on the original gold-label training dataset mentioned in:
https://arxiv.org/pdf/2305.14904.pdf{
"eval_samples": 86,
"test_f1": 0.8056872037914692,
"test_loss": 0.4573868425250922,
}
{
"DIRECT QUOTE_f1": 0.9284126278345932,
"full_f1": 0.8055235903337169,
"BACKGROUND_f1": 0.770718232044199,
"NO QUOTE_f1": 0.0,
"INDIRECT QUOTE_f1": 0.8142250530785562,
"PUBLISHED WORK_f1": 0.8680851063829788,
"STATEMENT_f1": 0.9477124183006536,
"PRESS REPORT_f1": 0.9457364341085273,
"DECLINED COMMENT_f1": 0.9444444444444444,
"SOCIAL MEDIA POST_f1": 0.8936170212765957,
"PROPOSAL/ORDER/LAW_f1": 0.5616438356164384,
"PRICE SIGNAL_f1": 0.7272727272727273,
"NARRATIVE_f1": 0.8833333333333333,
"DIRECT OBSERVATION_f1": 0.4324324324324324,
"COMMUNICATION_f1": 0.9672131147540983,
"PUBLIC SPEECH_f1": 0.983050847457627,
"VOTE/POLL_f1": 0.8205128205128205,
"COURT PROCEEDING_f1": 0.9491525423728813
}
