Math Solution Verification Classifier
Author: Hussein Adeiza (mabera)
Role: Licensed Environmental Health Officer, Abuja Nigeria
Base Model: Mixtral 8x7B
Fine-tuned with: AutoScientist by Adaption Labs
Model Description
A LoRA adapter fine-tuned to verify whether a proposed answer to a real
competition math problem is correct, classifying it as Correct or
Incorrect. This is a genuinely global-scope submission (not Nigeria-
specific), addressing a universal AI capability question: can a model
reliably grade mathematical correctness?
Training Data
- Source: MATH dataset (Hendrycks et al., NeurIPS 2021), accessed via
a properly-cited GitHub derivative (rasbt/math_full_minus_math500),
downloaded directly, 12,000 real competition problems
- Dataset: 20 rows (10 real problems x 2 answer variants each: one
correct, one deliberately perturbed incorrect answer, disclosed as
a constructed perturbation, not a real student error)
- Kaggle: https://www.kaggle.com/datasets/yunusahusseinadeiza/math-solution-verification-classifier
Important Note: Column Selection Correction
During training setup, the platform defaulted to training on
"Enhanced completion" text, which had drifted away from the closed-
label Correct/Incorrect structure into generic step-by-step tutoring
language, losing the classification task entirely. This was caught
and manually corrected by selecting "Original completion" instead
before training. Worth flagging for other builders working on
closed-label tasks: check which completion column is actually
selected before training, since the platform default may not be
the one you expect.
Training Metrics
- Win rate (on dataset): 76% adapted vs 24% base model
- Base model: mistralai/Mixtral-8x7B-Instruct-v0.1
- Method: LoRA (confirmed via training config), no recipe modifications
- Dataset quality: 7.0 → 9.0 (+28.6% relative improvement, Grade A)
- Percentile: 33.0
- Domain classification: Math (100%), a clean, accurate match
Verification
All 20 rows independently, programmatically verified before training:
every Correct/Incorrect classification checked against the real
ground-truth answer looked up directly in the raw MATH dataset source
file. 20/20 pass rate, demonstrated live in the accompanying Kaggle
notebook.
Why This Result Matters
This is the highest quality grade (A) and highest quality score
improvement (+28.6%) in this author's 18-submission AutoScientist
portfolio, achieved using the same disciplined closed-label methodology
(no recipe modifications, deterministic ground truth, independent
verification) established across the AMR and Loan Classifier
submissions, applied here to a genuinely global rather than
Nigeria-specific problem.
Credits
Powered by Adaptive Data — Adaption Labs
AutoScientist Challenge 2026, Part 2 — Math & Code Category
