Tos-Roberta is a fine-tuned RoBERTa-large model designed to classify clauses in Terms of Service (ToS) documents based on their fairness level. The model categorizes clauses into three classes: clearly fair, potentially unfair, and clearly unfair.
Task
The model performs multi-class classification on individual sentences or clauses, categorizing them into three levels of unfairness:
Clearly Fair
Potentially Unfair
Clearly Unfair
Key Features
Based on the RoBERTa-large architecture
Fine-tuned on a specialized dataset of ToS clauses
Achieves high accuracy in distinguishing between fair and unfair clauses
Suitable for legal text analysis and consumer rights applications
Performance
The model demonstrates strong performance on the task of ToS clause classification:
Validation Accuracy: 89.64%
Test Accuracy: 85.84%
Detailed performance metrics per epoch:
Epoch
Training Loss
Validation Loss
Accuracy
F1 Score
Precision
Recall
1
0.443500
0.398950
0.874699
0.858838
0.862516
0.874699
2
0.416400
0.438409
0.853012
0.847317
0.849916
0.853012
3
0.227700
0.505879
0.896386
0.893325
0.891521
0.896386
4
0.052600
0.667532
0.891566
0.893167
0.895115
0.891566
5
0.124200
0.747090
0.884337
0.887412
0.891807
0.884337
Training Details
Base Model: RoBERTa-large
Dataset: CodeHima/TOS_DatasetV2
Training Time: 3310.09 seconds
Epochs: 5
Batch Size: 8
Learning Rate: Started at 2e-5 with a warmup period and decay
Optimizer: AdamW
Loss Function: Cross-Entropy
Training Strategy: Mixed precision training (FP16)
Usage
To use this model for inference:
python
1from transformers import RobertaTokenizer, RobertaForSequenceClassification
2import torch
34# Load model and tokenizer5model = RobertaForSequenceClassification.from_pretrained("YourHuggingFaceUsername/Tos-Roberta")6tokenizer = RobertaTokenizer.from_pretrained("YourHuggingFaceUsername/Tos-Roberta")78# Prepare input text9text ="Your Terms of Service clause here"10inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=512)1112# Make prediction13with torch.no_grad():14 outputs = model(**inputs)1516probabilities = torch.softmax(outputs.logits, dim=-1)17predicted_class = torch.argmax(probabilities, dim=-1).item()1819# Map prediction to label20label_map ={0:"clearly_fair",1:"potentially_unfair",2:"clearly_unfair"}21predicted_label = label_map[predicted_class]2223print(f"Predicted class: {predicted_label}")24print(f"Probabilities: {probabilities[0].tolist()}")
Limitations and Bias
The model's performance may vary depending on the legal jurisdiction and specific domain of the ToS.
It may not capture nuanced legal interpretations that require human expertise.
The training data may contain biases present in existing ToS documents.
Ethical Considerations
While this model can assist in identifying potentially unfair clauses in ToS documents, it should not be used as a substitute for professional legal advice. The model's predictions should be reviewed by qualified legal professionals before making any decisions based on its output.
Citation
If you use this model in your research or application, please cite it as:
@misc{Tos-Roberta,
author = {Himanshu Mohanty},
title = {Tos-Roberta: RoBERTa-large model for Terms of Service Fairness Classification},
year = {2024},
publisher = {HuggingFace},
journal = {CodeHima/Tos-Roberta},
howpublished = {\url{https://huggingface.co/CodeHima/Tos-Roberta}}
}
Contact
For questions, feedback, or collaborations, please open an issue on the model's Hugging Face repository or contact [Your Contact Information].
