You can deploy this model and start earning money today!

Ef05/
LLM-Generated-Summaries-tdidf-baseline

Name: LLM-Generated-Summaries-tdidf-baseline
Availability: InStock

Views

No views yet

TF‑IDF baseline (AI vs Human summaries)

This repo trains a simple, reproducible TF‑IDF baseline to classify summaries as:

human (0)
ai (1)

It’s meant as a strong “sanity check” baseline for dissertation-style experiments: quick to run, easy to explain, easy to reproduce.

Quickstart

From the repo root:

python -m venv .venv

Activate the venv:

bash

# Windows (PowerShell)
.\.venv\Scripts\Activate.ps1

# macOS / Linux
source .venv/bin/activate

Install deps:

bash

python -m pip install -U pip
python -m pip install -r requirements.txt

Run one end-to-end experiment (discover → load → clean → split → train → eval → save):

python scripts/run_tfidf_baseline.py train --run-name tfidf_char35_logreg

If your data is split into separate Human / AI CSVs

You can pass --data multiple times to merge CSVs before training:

bash

python scripts/run_tfidf_baseline.py train --run-name tfidf_merged ^
  --data data/processed/human-generated/english-train-data.csv ^
  --data data/processed/ai-generated/english-ai-data.csv

Evaluate a saved model

Evaluate on the saved test split for the same run:

python scripts/run_tfidf_baseline.py eval --model-dir models/tfidf/tfidf_char35_logreg --run-name tfidf_char35_logreg

Dataset discovery (helps when you have lots of CSVs)

List CSV candidates under ./data (ranked by schema match):

python scripts/dataset_info.py

Run all (language, provider) pairs

If your repo follows the naming conventions under data/processed/ (see src/pairing.py), you can sweep:

python scripts/run_all_pairs.py --run-prefix sweep_char35_logreg --analyzer char --ngram-min 3 --ngram-max 5 --model logreg

What columns does the baseline expect?

At minimum, each input CSV needs:

id
language (normalized to: en, es, fr, ar, zh)
source_type (human or ai)
summary (or whichever column you pass as --text-col)

The pipeline creates:

label: human -> 0, ai -> 1
input_text: cleaned text used for TF‑IDF
group_id: used for group-aware splitting (prefers url if present, else id)

Outputs

Each --run-name <run> writes:

Splits: data/processed/<run>_train.csv, <run>_val.csv, <run>_test.csv
Model: models/tfidf/<run>/model.joblib, vectorizer.joblib, config.json
Reports: reports/<run>/report.md, metrics.json
- confusion_matrix.png is also saved if matplotlib is installed (optional dependency)

Common gotchas / troubleshooting

“dataset contains only one class label”: you probably trained on only Human or only AI data. Pass both (use repeated --data ...).
“Missing required columns”: your CSV schema doesn’t match. Run python scripts/dataset_info.py and/or rename columns.
“Multiple valid CSV datasets found”: the auto-discovery can’t guess which to use — pass --data <path> or --data-index <N>.

Ef05/LLM-Generated-Summaries-tdidf-baseline

TF‑IDF baseline (AI vs Human summaries)

Quickstart

If your data is split into separate Human / AI CSVs

Evaluate a saved model

Dataset discovery (helps when you have lots of CSVs)

Run all (language, provider) pairs

What columns does the baseline expect?

Outputs

Common gotchas / troubleshooting

Ef05/
LLM-Generated-Summaries-tdidf-baseline