Measured on a held-out test split of 6,899 images that took no part in training or checkpoint selection.
Metric
Test (held out)
Validation (selected the checkpoint)
Top-1 accuracy
98.68%
98.87%
Balanced accuracy
98.75%
98.80%
Macro F1
0.9864
0.9876
Top-5 accuracy
99.99%
100.00%
95% confidence interval on the test figure: 98.68% +/- 0.27, so [98.41, 98.95]. Quote it as "about 98.7%".
Why the evaluation protocol matters here
The Kaggle build of PlantVillage (new-plant-diseases-dataset) ships train/ and valid/ folders that are an 80/20 split of an already augmented image pool. Augmentation happened before the split, so rotations and flips of the same physical leaf land on both sides of it:
Measured on the shipped folders: 9,066 of 14,732 distinct source images in valid/ (61.5%) also appear in train/, and 7 of the 38 classes have no clean validation image at all.
This model was therefore trained on a rebuilt split that groups every file by the source leaf it came from and splits the groups, never the files. Training keeps every augmented copy of its own leaves; validation and test keep one file per leaf, so a single leaf is scored once rather than up to fifteen times. The split builder verifies the property rather than assuming it, and fails loudly if any source image reaches two splits.
An earlier version of this project reported 99.07%, which was a validation score on the leaked split. The honest figure on a clean held-out split is 98.68%, a drop of 0.39 points. The leakage was real; it barely moved the headline. The model was already good, and the protocol was what needed fixing.
AUC-ROC is deliberately not quoted as a headline. The clean test split scores 0.9999 on it, essentially the same as the leaked split did. At this accuracy across 38 well-separated classes a near-perfect macro OVR AUC is simply what the metric does, so it cannot distinguish a leaked evaluation from a good classifier.
Training
Phase
Epochs
Layers trained
LR (backbone / head)
Warm-up
5
Head only
- / 1e-3
Fine-tune
35
Entire network
1e-5 / 1e-3
AdamW, CosineAnnealingLR, batch size 64, 224x224 input, mixed precision on CUDA. Class-weighted cross-entropy on the training objective only; validation loss is unweighted so it stays comparable.
This checkpoint is epoch 34, selected on validation accuracy. Early stopping did not fire: the run reached the 40-epoch cap because a marginal new best at epoch 29 reset the patience counter. The curve is flat rather than still climbing, but the run was ended by a cap and not by a convergence criterion.
18,486,126 parameters.
Usage
python
1import torch
2from huggingface_hub import hf_hub_download
34path = hf_hub_download("Khawajaa/plant-disease-detector","best_model.pth")5ckpt = torch.load(path, map_location="cpu")6# ckpt keys: model_state_dict, epoch, val_acc, config78# The architecture is a custom head on torchvision's efficientnet_b4.9# Use the class from the repo so the state dict matches:10# git clone https://github.com/khawaja1447/plant-disease-detector11# from src.model import EfficientNetB4Classifier12# model = EfficientNetB4Classifier(num_classes=38, pretrained=False)13# model.load_state_dict(ckpt["model_state_dict"])
Inputs are 224x224 RGB, ImageNet-normalised (mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]). Scale and centre-crop rather than squashing off-square photos: every training image is square, so an anisotropic resize is a train/serve mismatch.
Limitations
Laboratory conditions. PlantVillage images are single detached leaves on plain backgrounds under even lighting. Field performance with occlusion, mixed infections, variable lighting and leaves still attached to a plant will be lower, and this model has not been measured on that.
Thin classes. Per-class test sizes range from 41 to 379 images. The smallest classes carry a wide confidence interval.
Weakest classes on test: Tomato target spot (87.20%), corn gray leaf spot (90.91%), corn northern leaf blight (95.95%), tomato early blight (96.00%). Target spot, early blight and late blight on tomato are confused with each other, which is a genuine diagnostic difficulty rather than a modelling artefact.
Closed set. The model always returns one of 38 classes. Given a species it was not trained on, or a non-leaf image, it will still produce a confident-looking prediction.
Not agronomic advice. The demo app pairs predictions with general treatment guidance and a disclaimer. Product registration varies by country and changes over time. Confirm any diagnosis with a certified agronomist and check local product registration before applying anything.
