EU-AI-Act-Aware Resume-Job Fit Classifier
A two-stage resume-to-job-description fit classifier, built with the EU AI Act's
transparency and bias-disclosure requirements in mind. Recruitment scoring/ranking
tools are explicitly treated as high-risk AI systems under the Act's draft guidance,
so this model prioritizes honest disclosure of its limitations over polished-looking
performance numbers.
Two-stage design
- Stage 1 (reliable): No Fit vs Some Fit — 66% accuracy (vs. 51% majority baseline),
balanced across both classes.
- Stage 2 (weak, use with caution): Good Fit vs Potential Fit — only 57% accuracy
(vs. 51% baseline) when applied to "Some Fit" pairs. This distinction should be
treated as a soft hint, never a standalone decision.
This split exists because a single 3-class model conflated a distinction the data
does not clearly support (Good vs Potential Fit) with one it does (Fit vs No Fit).
Forcing false confidence on the former would have been misleading.
What this model outputs
For each resume/job description pair:
- A primary Fit / No Fit prediction with confidence
- The resume-job semantic similarity score
- The top words/features driving the decision (explainability)
- A secondary Good/Potential signal, explicitly flagged as low-reliability
Known limitations (disclosed deliberately)
- Truncation was not the fix — reframing was. An earlier transformer-based
approach (DistilBERT) performed at or below baseline (47%) because resumes
average 1,207 tokens, far exceeding the 512-token limit, silently cutting off
later resume sections (often the Skills section). A TF-IDF + similarity-feature
approach was used instead, since it is not token-limited.
- Spurious correlations were found and partially fixed. An earlier version of
this model relied partly on company/organization names (e.g. specific employers)
as predictive features. Named-entity stripping (spaCy NER) removed most of this,
with accuracy dropping only 1 point (67% -> 66%), confirming the model's real
signal was not primarily coming from these artifacts.
- A residual bias-shaped pattern remains unresolved. Certain legitimate skill
words (e.g. "electrical", "engineering", "data", "analytics") still correlate
with "No Fit" predictions. This may reflect genuine class imbalance in which job
categories are underrepresented among "Fit" labels in the training data, rather
than a universal truth about those skills. This has NOT been fully root-caused
and should be investigated further before any production use.
- This model should not make autonomous hiring decisions. Consistent with the
EU AI Act's human-oversight requirement for high-risk employment AI systems, all
outputs should be reviewed by a human recruiter, and no candidate should be
rejected solely on this model's output.
Intended use
Portfolio / research demonstration of building an explainable, limitation-aware
resume-screening classifier. Not validated for production hiring decisions.
Training data
cnamuangtoun/resume-job-description-fit — 6,241 train / 1,759 test resume-job
pairs, labeled No Fit / Potential Fit / Good Fit.
Author
Syed Sibte Hassan Shah - International Recruiter & AI Automation Specialist,
applying EU AI Act / GDPR compliance background to practical NLP model-building.
