This model is used in OpenPlants (domai-tb/OpenPlants), an open-source and privacy-friendly companion app for plant care. Inside the app, it powers local image classification so users can identify plants directly on-device from a photo.

Built on google/vit-base-patch16-224, the model was fine-tuned for species-level plant identification on the GBIF / iNaturalist plant image dataset.

Highlights

Property	Value
Base model	`google/vit-base-patch16-224`
Parameters	~97.2M
Training samples	2,000,000 curated plant occurrences
Species coverage	~14,000 unique species
Source data	GBIF / iNaturalist
Training method	End-to-end supervised fine-tuning
Primary use	Fast plant species classification from a single image

OpenPlants

OpenPlants is the companion app this model was built for.

Runs plant identification locally
Keeps inference on-device for a privacy-friendly experience
Helps users identify plants in a fast, offline-friendly workflow

Example Usage

python

from transformers import AutoImageProcessor, AutoModelForImageClassification
from PIL import Image
import requests
import torch

model_id = "domai-tb/OpenPlants-Identification-ViT-Base-Patch16-224"

processor = AutoImageProcessor.from_pretrained(model_id)
model = AutoModelForImageClassification.from_pretrained(model_id)

url = "https://example.com/plant.jpg"
image = Image.open(requests.get(url, stream=True).raw)

inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits

probs = logits.softmax(dim=-1)[0]
topk = torch.topk(probs, k=5)

for prob, idx in zip(topk.values, topk.indices):
    label = model.config.id2label[idx.item()]
    print(f"{label}: {prob.item():.4f}")

Intended Applications

Plant identification in mobile apps
Ecological surveys
Nursery and horticulture tools
Restoration and revegetation workflows
Field research and biodiversity monitoring
Citizen science and educational platforms
Image-based species tagging pipelines

Data and Training Details

Dataset Construction

Sourced from GBIF / iNaturalist occurrences with valid species and image metadata
Cleaned and deduplicated before training
Species filtered to those with at least 20 images
Maximum cap of 1,000 images per species to reduce class imbalance
Final training dataset: 2,000,000 images across roughly 14,000 species

Training

ViT-Base fine-tuned for 1 epoch over 2M samples
AdamW optimizer with standard ViT augmentations
Mixed-precision training on GPU

Limitations

Some species are visually indistinguishable without context such as location or reproductive structures
Performance varies for rare, morphologically similar, or poorly photographed species
The model is purely image-based and does not use location metadata

Labels

Species names follow canonical GBIF taxonomy (species_name). Each class maps directly to one species.

You can inspect all labels with:

python

from transformers import AutoConfig

cfg = AutoConfig.from_pretrained("domai-tb/OpenPlants-Identification-ViT-Base-Patch16-224")
labels = cfg.id2label

Fine-Tuning and Adapters

You can further specialize the model with LoRA adapters for:

Regional subsets
Functional groups
Threatened species
Agricultural crops
Disease classification

The base model is broad enough to support domain-specific adapter tuning with relatively little compute.

License

Model weights follow the same license as the underlying ViT-Base model. Users are responsible for complying with GBIF and iNaturalist usage terms for any downstream dataset creation.