EC_MODELS (AI vs Human Detection)
This repository contains model artifacts for AI‑generated text detection experiments, including a HOPE classifier and a stacked ensemble (DeBERTa + LightGBM + TF‑IDF/SGD). It is not a single model, but a collection of checkpoints and meta‑models produced during experiments.
Model inventory
Artifacts are stored in src/ai_vs_human/models/:
hope/
HOPE checkpoints (fold_*/best_model.pt) for a transformer‑based classifier with memory modules.deberta_v3_base/
DeBERTa base checkpoints used in the ensemble.lightgbm/, lightgbm_numeric/
LightGBM models on engineered features.tfidf_sgd/
TF‑IDF + SGD models.stack_meta/meta_learner.joblib
Logistic‑regression meta‑learner used for stacking.
Out‑of‑fold predictions used to compute metrics and train the stacker are in:
src/ai_vs_human/oof/ (e.g., oof_stack.csv, oof_deberta.csv, oof_lgb.csv, oof_sgd.csv)
Intended use
These models are intended for research and evaluation of AI‑generated text detection. They can be used to:
- compare HOPE vs. ensemble baselines,
- reproduce experiments from the notebooks,
- evaluate domain‑shift and multilingual robustness.
How to evaluate (metrics only)
Use the metrics‑only notebook to compute standard metrics without retraining:
src/ai_vs_human/metrics_only.ipynb
This notebook loads oof_stack.csv and prints AUC‑ROC, PR‑AUC, Accuracy, Precision, Recall, F1, Brier, and ECE, plus a best‑F1 threshold.
Training data
Training data and features were produced from the project’s datasets under src/ai_vs_human/, including:
merged_ai_human_multisocial_features*.csv
See the dataset‑building and training notebooks for details:
src/ai_vs_human/ai_generated_text_detection.ipynbsrc/ai_vs_human/hope_train_distributed.py
Evaluation notes
Metrics depend on threshold selection and domain. This repo includes tools for:
- internal test evaluation,
- external/ood evaluation,
- calibration and threshold selection.
See:
src/ai_vs_human/evaluation_suite.ipynb
Limitations and biases
- Performance can degrade under domain shift (new sources or languages).
- False‑positive rates for human multilingual text can be high without careful calibration.
- These models are not a definitive AI‑detection system and should not be used for high‑stakes decisions without additional validation.
License
No explicit license is included in this repo. Please add a license if you intend to distribute or reuse the models.
