You can deploy this model and start earning money today!

StanfordSCALE/
tutor_talkmoves_modernbert_base_classifier

Name: tutor_talkmoves_modernbert_base_classifier
Availability: InStock

text-embeddings-inference

endpoints_compatible

Views

No views yet

Tutor Talk Moves Classifier (ModernBERT)

Model Description

This model is a fine-tuned version of answerdotai/ModernBERT-base that classifies tutor utterances based on teacher talk moves from talkmoves.com. The model identifies whether a tutor's message represents one of the following three categories, or none of them:

Classroom management
Pressing for accuracy or reasoning
Restating or revoicing

Training Data

This model is trained on text of tutoring sessions from three different tutoring providers, annotated by two raters.

Inter-Rater Reliability with Krippendorff's Alpha: 0.83
Total Training Examples: 1,849
Class Distribution: | Class | Examples | Percentage | |-------|----------|------------| | None | 987 | 53.4% | | Classroom management | 284 | 15.4% | | Pressing for accuracy or reasoning | 474 | 25.6% | | Restating or revoicing | 104 | 5.6% |

Data Format

The is trained on utterances with the following format: [PRETEXT] {3 previous messages} [TEXT] {target message}, where [PRETEXT] and [TEXT] are special tokens. Names are anonymized, message text is lowercased, and leading and trailing whitespace is removed.

Example:

[PRETEXT] tutor: hello there [student]
tutor: what is the answer to this problem?
student: the answer is 6 [TEXT] tutor: why do you say the answer is 6?

Performance

Test set results (264 examples):

Class	Precision	Recall	F1-Score	Support
0 - None	0.9158	0.9062	0.9110	192
1 - Classroom Management	0.6818	0.6818	0.6818	22
2 - Pressing for Accuracy or Reasoning	0.7826	0.8571	0.8182	42
3 - Restating or Revoicing	0.5000	0.3750	0.4286	8
Macro Average	0.7201	0.7051	0.7099	264
Weighted Average	0.8625	0.8636	0.8625	264

StanfordSCALE/tutor_talkmoves_modernbert_base_classifier