RoBERTa-Large CV AI Detection Model
Model Overview
This model is a fine-tuned RoBERTa-Large transformer trained to detect
whether a curriculum vitae (CV) was written by a human, generated by AI,
or produced through a combination of both.
The model was trained using the dataset:
jamal-ibrahim/ai-cv-detection-dataset
The task is formulated as a three-class classification problem:
Label Meaning
human CV written entirely by a human
mixed CV containing both human-written and AI-generated content
ai_generated CV generated primarily by an AI system
The goal of this experiment was to explore whether modern transformer
models can identify stylistic differences between human-written and
AI-generated professional documents.
Experimental Setup
Model Architecture
Base model:
roberta-large
RoBERTa-Large contains approximately 355 million parameters and is known
to perform well on tasks involving linguistic style, authorship
detection, and semantic classification.
Dataset
Dataset used for training:
jamal-ibrahim/ai-cv-detection-dataset
Total dataset size: approximately 1500 CV documents.
The dataset was split using a stratified strategy to maintain class
balance.
Split configuration:
Split Percentage
Train 70%
Validation 15%
Test 15%
Each class (human, mixed, ai_generated) is represented equally in each
split.
Training Configuration
Training parameters:
Parameter Value
Epochs 10
Batch size 16
Learning rate 5e-6
Optimizer AdamW
Max sequence length 512
Evaluation strategy per epoch
Training was performed using the Hugging Face Trainer API.
Results
Evaluation was performed on the held-out test set.
Confusion Matrix
True / Predicted Human Mixed AI
Human 75 0 0
Mixed 0 74 1
AI Generated 0 0 75
Only one misclassification occurred in the test set.
Metrics
Metric Value
Accuracy ~0.995
Weighted F1 ~0.996
Macro F1 ~0.995
The model demonstrates near-perfect classification performance on this
dataset.
Interpretation of Results
The extremely high classification accuracy suggests that the model
successfully learned stylistic patterns that differentiate:
- human-authored CVs
- AI-generated CVs
- hybrid documents
Several factors may contribute to this performance.
Structural Patterns in AI-Generated CVs
Large language models often produce documents with consistent structural
patterns such as:
- uniform sentence lengths
- repeated phrasing patterns
- symmetrical bullet point structures
- consistent grammatical style
These patterns may create detectable stylistic signatures.
Human Writing Variability
Human-written CVs often contain:
- irregular formatting
- inconsistent sentence structures
- varied vocabulary
- domain-specific phrasing
This variability may make human documents distinguishable from
AI-generated ones.
Mixed Documents as a Transitional Style
Documents labeled as mixed appear to occupy an intermediate stylistic
space between human and AI-generated text.
The single observed misclassification (mixed → AI-generated) suggests
that documents heavily edited by AI may resemble fully generated
content.
Limitations
Although the results are strong, several limitations should be
considered.
Dataset Size
The dataset contains approximately 1500 samples, which is relatively
small for training large transformer models.
High performance may partly reflect the limited diversity of the
dataset.
Potential Template Effects
If AI-generated CVs were produced using similar prompts or generation
patterns, the model may learn those specific templates rather than
generalizable AI-writing signals.
Domain Specificity
This model was trained specifically on CV-style documents and may not
generalize well to other types of text such as essays, emails, or
reports.
Future Work
Future experiments could explore several directions:
- cross-validation to measure performance stability
- testing on unseen CV datasets
- evaluating different LLM-generated CV styles
- reducing model size using distillation or quantization
- hybrid approaches combining stylistic features with transformer
embeddings
Intended Use
Potential applications include:
- research on AI-generated text detection
- analysis of AI assistance in professional writing
- experimentation with authorship attribution in structured documents
The model should not be used as a definitive tool for determining
authorship in real-world hiring decisions.
Citation
@model{roberta_large_cv_detector, title = {RoBERTa-Large CV AI Detection
Model}, author = {Jamal Ibrahim}, year = {2026}, dataset =
{jamal-ibrahim/ai-cv-detection-dataset}, publisher = {Hugging Face} }
