ReviewMate Rhetorical Sentence Classifier — v1
A DistilBERT model that classifies sentences from research paper abstracts into five rhetorical roles. Optimized for empirical scientific writing across STEM domains, with strong performance on computer science / AI literature.
Task
Given a sentence from a research abstract, predict its rhetorical role:
BACKGROUND — context, prior work, motivationOBJECTIVE — research goal or hypothesisMETHODS — approach, design, techniques usedRESULTS — findings, benchmarks, measurementsCONCLUSIONS — interpretation, implications, future work
These rhetorical categories apply to empirical scientific writing across most STEM domains.
Performance
Evaluated on held-out test sets:
| Test Set | Domain | Accuracy | F1 Macro | F1 Weighted |
|---|
| CSAbstruct | CS / AI | 0.7517 | 0.7556 | 0.7527 |
| PubMed-RCT | Biomedical | 0.8833 | 0.8294 | 0.8825 |
Best suited for computer science, AI/ML, applied STEM, and quantitative empirical research. Performance may degrade on theoretical, mathematical, or humanities papers due to different rhetorical conventions.
Usage
1from transformers import AutoTokenizer, AutoModelForSequenceClassification
2import torch
3
4model_id = "Himel000/reviewmate-classifier-v1"
5tokenizer = AutoTokenizer.from_pretrained(model_id)
6model = AutoModelForSequenceClassification.from_pretrained(model_id)
7
8sentence = "We propose a novel transformer architecture for sequence classification."
9inputs = tokenizer(sentence, return_tensors="pt", truncation=True, max_length=64)
10
11with torch.no_grad():
12 outputs = model(**inputs)
13
14predicted_class_id = outputs.logits.argmax(dim=-1).item()
15predicted_label = model.config.id2label[predicted_class_id]
16print(predicted_label)
Training Approach
The model was fine-tuned in two sequential stages to leverage large-scale rhetorical data while adapting to target domain writing style.
Stage 1 — Large-scale rhetorical learning:
- Base:
distilbert-base-uncased
- Data: PubMed-RCT-200k (~2.2M labeled sentences)
- Epochs: 2, learning rate: 5e-5, effective batch size: 128
- Purpose: Learn general rhetorical patterns from the largest available labeled dataset
Stage 2 — Domain adaptation:
- Continued fine-tuning from Stage 1 weights
- Data: CSAbstruct (~11k labeled CS sentences)
- Epochs: 4, learning rate: 2e-5 (low to preserve Stage 1 knowledge), batch size: 32
- Purpose: Adapt rhetorical patterns to CS/AI writing conventions
Total compute: ~4 hours (Kaggle T4 x2).
The two-stage approach improved out-of-domain F1 from 0.39 (after Stage 1 alone) to 0.76, demonstrating the value of staged transfer learning for cross-domain rhetorical classification.
Project Context
This model powers the rhetorical extraction component of
ReviewMate, an AI-powered literature scanning and synthesis tool for empirical research papers.
Scope and Limitations
- Optimized for empirical scientific writing (introduction → method → result → conclusion structure)
- Strongest on CS, AI/ML, applied STEM, and quantitative empirical research
- Not designed for theoretical mathematics (theorem-proof structure), humanities, or non-empirical writing
- Single-sentence classification (no surrounding sentence context)
- Cross-domain expansion to additional fields is part of the project's open contribution roadmap
Citation
Datasets used for training:
Dernoncourt, F., & Lee, J. Y. (2017). PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts. IJCNLP 2017.
Cohan, A., Beltagy, I., King, D., Dalvi, B., & Weld, D. (2019). Pretrained Language Models for Sequential Sentence Classification. EMNLP 2019.
