This model is a fine-tuned version of FacebookAI/roberta-base designed to detect whether a given text is generated by AI or written by a human. It has been sequentially fine-tuned distinct datasets to enhance its robustness and generalization capabilities across various text domains. The model aims to achieve high accuracy in distinguishing between human-produced and machine-generated content, making it a valuable tool for applications requiring AI text detection.

Intended Use: The model is intended for research and practical applications involving the detection of AI-generated text. It can be used as a component in systems for content moderation, academic integrity checks, or general content analysis.

Limitations and Bias:

While trained on diverse datasets, the model's performance may vary on text generated by new or unseen AI models, or on highly domain-specific text not well-represented in the training data.
Like all AI models, it may exhibit biases present in the training data. Further evaluation on diverse and representative datasets is recommended for critical applications.
The model's predictions should be interpreted as probabilities and used as one signal among others, not as absolute truth.

Training Data

The model was sequentially fine-tuned on the following datasets:

artem9k/ai-text-detection-pile:
- Subset Used: A total of 20,000 samples were utilized, specifically 10,000 rows from train-00000-of-00007-bc5952562e004d67.parquet and 10,000 rows from train-00006-of-00007-3d8a471ba0cf1c8d.parquet. This dataset contains a variety of text samples labeled as AI-generated or human-written.
Ateeqq/AI-and-Human-Generated-Text:
- Subset Used: All available samples from this dataset were used. This dataset provides additional diverse text samples for robust AI detection.
NicolaiSivesind/human-vs-machine:
- Subset Used: All available samples from this dataset were used, specifically from wiki-labeled.csv and research-abstracts-labeled.csv. This further enhances the model's ability to distinguish between human-produced and machine-generated content, particularly focusing on Wikipedia articles and research abstracts.

Training Procedure

Base Model

The model started with weights from FacebookAI/roberta-base.

Fine-tuning Strategy

The model was fine-tuned sequentially, meaning the output of one fine-tuning stage served as the starting point for the next.

Hyperparameters

The following hyperparameters were used during each fine-tuning phase:

Learning Rate: 2e-5
Per Device Train Batch Size: 8
Per Device Eval Batch Size: 8
Gradient Accumulation Steps: 2 (resulting in an effective batch size of 16)
Number of Train Epochs: 3 (per fine-tuning phase)
Weight Decay: 0.01

Hardware

All training was performed on a Google Colab GPU T4.

Training Times

Fine-tuning on artem9k/ai-text-detection-pile: 1 hour 12 minutes 14 seconds
Fine-tuning on Ateeqq/AI-and-Human-Generated-Text: 0 hours 30 minutes 36 seconds
Fine-tuning on NicolaiSivesind/human-vs-machine: 1 hour 04 minutes 22 seconds

Overall Total Training Time: Approximately 2 hours 47 minutes 12 seconds across all fine-tuning stages.

Evaluation

The model's performance was evaluated on the validation splits of each respective dataset during its fine-tuning phase. The overall metrics reflect the performance of the final model after all sequential fine-tuning stages.

Metrics

Eval Loss: 0.10
Eval F1 Score: 0.972
Train Loss: 0.20
Train Samples per Second: 32
Runtime (Evaluation): 33s

Usage

Inference

To use the model for inference, you can load it with the Hugging Face Transformers library:

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

# Define the model name/path
model_name = "MonkeyDAnh/my-awesome-ai-detector-roberta-base"

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Move model to GPU if available
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

# Define label mapping (ensure this matches your model's config.id2label if available)
# Based on your ClassLabel definition (0: human-produced, 1: machine-generated)
id_to_label = model.config.id2label if hasattr(model.config, 'id2label') else {0: "human-produced", 1: "machine-generated"}

# Example text for inference
text = "This is a sample text to detect AI generation. It is written to sound very robotic and unnatural."

# Tokenize the input text
# Using max_length=512 for standard RoBERTa, adjust if your MAX_LENGTH_CHUNK was different
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=512).to(device)

# Perform inference
outputs = model(**inputs)

# Get predicted label ID and convert to integer
predictions_id = outputs.logits.argmax(-1).item()

# Map ID to the human-readable label
predicted_label = id_to_label[predictions_id]

print(f"Text: \"{text}\"")
print(f"Predicted Label ID: {predictions_id}")
print(f"Predicted Label: {predicted_label}")
# Expected output examples:
# Predicted Label: human-produced
# Predicted Label: machine-generated

MonkeyDAnh/my-awesome-ai-detector-roberta-base-v4-human-vs-machine-finetune

my-awesome-ai-detector-roberta-base

Model Description