MLM Classifier turns a small labeled text dataset into a tested, downloadable local classifier. A user confirms every label and every public document before it enters the dataset. Candidate models are evaluated on a fixed holdout set and promoted only when macro F1 improves.
The application is domain-independent. Included label profiles cover annual reports, banking credit, insurance claims, contracts, and support tickets.
Fastest test
Open the app and click Load annual-report example.
Open 2. Label, confirm or change a suggested label, and save it.
Open 3. Train & test and train a candidate model.
Paste text into Classify locally to run inference with the promoted model.
Open 4. Export & API to download both the dataset and standalone model bundles.
This path performs real scikit-learn fitting, evaluation, serialization, and inference. It does not require an OpenAI API key.
Run locally
Python 3.10 or newer is required. No PowerShell activation is needed.
Windows
From the repository folder, double-click run.cmd. On the first run it creates the environment and installs the application automatically. Or use PowerShell:
The included annual-report project trains an initial local TF-IDF and logistic-regression model. A custom project accepts labeled seed JSONL, labeled evaluation JSONL, unlabeled JSONL, and either an included or custom label profile.
Label and acquire data
MLM shows one useful example, model probabilities, and available labels. Only the user's confirmed label enters training data. The acquisition planner can prioritize uncertainty, a rare class, diversity, a hard negative, a confusion pair, or profile terms.
OTHER is an explicit class. By default, MLM requests at least one OTHER hard negative for every two positive examples, which helps prevent a classifier from learning only the common target labels.
Public-document search is optional. Search results are suggestions; MLM fetches nothing until the user selects exact sources and checks the confirmation box.
Train and evaluate
Each retraining run fits a new local classifier using seed and confirmed replay labels, evaluates it on the fixed holdout set, registers an immutable candidate, and promotes it only when macro F1 improves. A weaker candidate remains in the audit trail and never replaces production.
Export and serve
The app creates downloadable dataset and model ZIP files with manifests and SHA-256 checksums. The model bundle contains a standalone prediction script. A local installation can also start a FastAPI endpoint with:
GET /health
GET /model
POST /predict
POST /feedback
Endpoint feedback enters an in-app review queue. It cannot alter the dataset or model until a user approves it and separately retrains.
Input formats
Seed and evaluation rows require text and label. Unlabeled rows require text. Source information and other metadata are optional.
{"text":"Termination may occur with 30 days notice.","label":"TERMINATION","source_document":"contract-17","page":8}
A label profile is plain JSON:
json
1{2"name":"Contract clause classification",3"description":"Classify substantive contract clauses.",4"labels":{5"TERMINATION":"Rights or conditions to terminate the agreement.",6"PAYMENT":"Fees, invoices, payment dates, or late charges.",7"OTHER":"Text outside the target classes."8},9"discovery_terms":["terminate","notice","invoice","payment"]10}
OpenAI features
An OPENAI_API_KEY is needed only for optional GPT-5.6 strategy selection, selective teaching through the CLI, and public web discovery. Local planning, labeling, training, evaluation, inference, export, and endpoint serving work without it.
MLM reads the key from the process environment. The UI never displays or saves it, and .env files are excluded from Git.
Codex was used to turn the existing classification research code into this portable product: the guided Gradio UI, project lifecycle, human-confirmation gates, evaluation and promotion flow, exports, endpoint feedback review, test suite, and deployment packaging. At runtime, GPT-5.6 can propose an acquisition strategy or act as a selective teacher. Its output is validated data; deterministic application code controls fetching, label admission, training, evaluation, and deployment.
The deterministic suite covers real fitting and serialization, promotion and rejection, class-balance enforcement, confirmation-gated web acquisition, portable exports, endpoint inference, and feedback review. It makes no API calls.
Safety boundaries
Holdout examples never enter training.
GPT output must pass schema, label, confidence, evidence, uniqueness, and conflict checks.
GPT strategy output is validated data, never executable code.
Search and fetch are separate actions; fetching requires explicit confirmation.
Web fetching blocks local/private network targets and limits content type and response size.
Model candidates are immutable after registration.
Feedback, training, and promotion are separate visible actions.
Human review remains necessary for consequential decisions.
