You can deploy this model and start earning money today!

stealthcode/
ai-detection

binary-classification

deberta-v3-large

text-classification

stealthcode/ai-detection

microsoft/deberta-v3-large

adapter

model-index

Views

No views yet

AI Detector LoRA (DeBERTa-v3-large)

LoRA adapter for binary AI-text vs Human-text detection, trained on ~2.7M English samples (label: 1 = AI, 0 = Human) using microsoft/deberta-v3-large as the base model.

Base model: microsoft/deberta-v3-large
Task: Binary classification (AI vs Human)
Head: Single-logit + BCEWithLogitsLoss
Adapter type: LoRA (peft)
Hardware: 8 x RTX 5090, bf16, multi-GPU
Final decision threshold: 0.8697 (max-F1 on calibration set)

Files in this repo

adapter/ – LoRA weights saved with peft_model.save_pretrained(...)
merged_model/ – fully merged model (base + LoRA) for standalone use
threshold.json – chosen deployment threshold and validation F1
calibration.json – temperature scaling parameters and calibration metrics
results.json – hyperparameters, validation threshold search, test metrics
training_log_history.csv – raw Trainer log history
predictions_calib.csv – calibration-set probabilities and labels
predictions_test.csv – test probabilities and labels
figures/ – training and evaluation plots
README.md – this file

Metrics (test set, n=279,241)

Using threshold 0.8697:

Metric	Value
AUROC	0.9985
Average Precision (AP)	0.9985
F1	0.9812
Accuracy	0.9814
Precision (AI)	0.9902
Recall (AI)	0.9724
Precision (Human)	0.9728
Recall (Human)	0.9904

Confusion matrix (test):

True Negatives (Human correctly): 138,276
False Positives (Human → AI): 1,345
False Negatives (AI → Human): 3,859
True Positives (AI correctly): 135,761

Calibration

Method: temperature scaling
Temperature (T): 1.4437
Calibration set: calibration
Test ECE: 0.0075 → 0.0116 (after calibration)
Test Brier: 0.0157 → 0.0156 (after calibration)

Plots

Training & validation

Learning curves:

Learning curves
Eval metrics over time:

Eval metrics

Validation set

ROC:

ROC (calib)
Precision–Recall:

PR (calib)
Calibration curve:

Calibration (calib)
F1 vs threshold:

F1 vs threshold (calib)

Test set

ROC:

ROC (test)
Precision–Recall:

PR (test)
Calibration curve:

Calibration (test)
Confusion matrix:

Confusion matrix (test)

Usage

Load base + LoRA adapter

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
from peft import PeftModel
import torch
import json

base_model_id = "microsoft/deberta-v3-large"
adapter_id    = "stealthcode/ai-detection"  # or local: "./adapter"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)

base_model = AutoModelForSequenceClassification.from_pretrained(
    base_model_id,
    num_labels=1,  # single logit for BCEWithLogitsLoss
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

Inference with threshold

python

# load threshold
with open("threshold.json") as f:
    thr = json.load(f)["threshold"]  # 0.8697

def predict_proba(texts):
    enc = tokenizer(
        texts,
        padding=True,
        truncation=True,
        max_length=512,
        return_tensors="pt",
    )
    with torch.no_grad():
        logits = model(**enc).logits.squeeze(-1)
        probs = torch.sigmoid(logits)
    return probs.cpu().numpy()

def predict_label(texts, threshold=thr):
    probs = predict_proba(texts)
    return (probs >= threshold).astype(int)

# example
texts = ["Some example text to classify"]
probs = predict_proba(texts)
labels = predict_label(texts)
print(probs, labels)  # label 1 = AI, 0 = Human

Load merged model (no PEFT required)

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch, json

model_dir = "./merged_model"
tokenizer = AutoTokenizer.from_pretrained(model_dir)
model = AutoModelForSequenceClassification.from_pretrained(model_dir)
model.eval()

with open("threshold.json") as f:
    thr = json.load(f)["threshold"]  # 0.8697

def predict_proba(texts):
    enc = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
    with torch.no_grad():
        logits = model(**enc).logits.squeeze(-1)
        probs = torch.sigmoid(logits)
    return probs.cpu().numpy()

Optional: apply temperature scaling to logits

python

import json
with open("calibration.json") as f:
    T = json.load(f)["temperature"]  # e.g., 1.4437

def predict_proba_calibrated(texts):
    enc = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
    with torch.no_grad():
        logits = model(**enc).logits.squeeze(-1)
        probs = torch.sigmoid(logits / T)
    return probs.cpu().numpy()

Notes

Classifier head is trainable together with LoRA layers (unfrozen after applying PEFT).
LoRA config:
- r=32, alpha=128, dropout=0.0
- Target modules: query_proj, key_proj, value_proj
Training config:
- bf16=True
- optim="adamw_torch_fused"
- lr_scheduler_type="cosine_with_restarts"
- num_train_epochs=2
- per_device_train_batch_size=8, gradient_accumulation_steps=4
- max_grad_norm=0.5
Threshold 0.8697 was chosen as the max-F1 point on the calibration set. You can adjust it if you prefer fewer false positives or fewer false negatives.