Epochs: 4
Best Validation Loss: 4.7171
Fine-tuning Strategy: Light fine-tuning with regularization to preserve generalization
Regularization: MSE loss to keep embeddings close to original CLIP

Classes

The model was trained on the following plant disease classes:

Apple___Apple_scab
Apple___Black_rot
Apple___Cedar_apple_rust
Apple___healthy
Blueberry___healthy
Cherry_(including_sour)___Powdery_mildew
Cherry_(including_sour)___healthy
Corn_(maize)___Cercospora_leaf_spot Gray_leaf_spot
Corn_(maize)__Common_rust
Corn_(maize)___Northern_Leaf_Blight
Corn_(maize)___healthy
Grape___Black_rot
Grape___Esca_(Black_Measles)
Grape___Leaf_blight_(Isariopsis_Leaf_Spot)
Grape___healthy
Orange___Haunglongbing_(Citrus_greening)
Peach___Bacterial_spot
Peach___healthy
Pepper,_bell___Bacterial_spot
Pepper,_bell___healthy
Potato___Early_blight
Potato___Late_blight
Potato___healthy
Raspberry___healthy
Soybean___healthy
Squash___Powdery_mildew
Strawberry___Leaf_scorch
Strawberry___healthy
Tomato___Bacterial_spot
Tomato___Early_blight
Tomato___Late_blight
Tomato___Leaf_Mold
Tomato___Septoria_leaf_spot
Tomato___Spider_mites Two-spotted_spider_mite
Tomato___Target_Spot
Tomato___Tomato_Yellow_Leaf_Curl_Virus
Tomato___Tomato_mosaic_virus
Tomato___healthy

Usage

python

import torch
import open_clip
from PIL import Image

# Load the model
model, _, preprocess = open_clip.create_model_and_transforms('ViT-B-32', pretrained='openai')
checkpoint = torch.load('model.pt', map_location='cpu')
model.visual.load_state_dict(checkpoint['vision_encoder'])
model.eval()

# Process an image
image = preprocess(Image.open('plant_image.jpg')).unsqueeze(0)
with torch.no_grad():
    image_features = model.visual(image)
    image_features = image_features / image_features.norm(dim=-1, keepdim=True)

# Use embeddings for similarity search in your vector database

Intended Use

This model is designed for:

Embedding plant disease images for similarity search
Building a vector database for plant disease diagnosis
Assisting farmers in identifying crop diseases

Limitations

Trained on 38 specific plant disease classes
Performance on completely unseen plant types may vary
Should be used as a decision support tool, not a replacement for expert diagnosis

Training Data

The model was fine-tuned on the Plant Disease Dataset containing images of:

Apple, Blueberry, Cherry, Corn, Grape, Orange, Peach, Pepper, Potato, Raspberry, Soybean, Squash, Strawberry, and Tomato plants
Various disease conditions and healthy samples

Citation

If you use this model, please cite the original CLIP paper:

@inproceedings{radford2021learning,
  title={Learning Transferable Visual Models From Natural Language Supervision},
  author={Radford, Alec and Kim, Jong Wook and Hallacy, Chris and Ramesh, Aditya and Goh, Gabriel and Agarwal, Sandhini and Sastry, Girish and Askell, Amanda and Mishkin, Pamela and Clark, Jack and others},
  booktitle={International Conference on Machine Learning},
  pages={8748--8763},
  year={2021},
  organization={PMLR}
}

hiimivantang/plant-disease-clip

Plant Disease CLIP Model (Fine-tuned ViT-B-32)

Model Details

Training Details