UCSCPhytoViT83 is a Vision Transformer (ViT) model fine-tuned for image classification of phytoplankton species using labeled images collected from the Imaging FlowCytobot (IFCB) at UCSC. The model is fine-tuned from the pre-trained google/vit-base-patch16-224-in21k base model. The model was trained on images that are aggregated from IFCB104, IFCB161, and IFCB116

Developed by: Patrick Daniel
Model type: Vision Transformer for image classification
License: Apache 2.0
Finetuned from model: google/vit-base-patch16-224-in21k

Model Sources

Repository: [More Information Needed]

Uses

Direct Use

This model can be used directly for classifying phytoplankton images captured by Imaging FlowCytobots instruments. Focus has been on capturing the variability of the phytoplankton community in Monterey Bay, CA, USA. It is intended for researchers.

Images should be transformed before inference:

python

    transforms.Compose([
    transforms.Resize((224, 224)),  # match ViT input size
    transforms.Normalize(mean=(0.485, 0.456, 0.406),
                         std=(0.229, 0.224, 0.225))
])

Out-of-Scope Use

This model is not intended for classifying non-phytoplankton images or images from different microscopy systems without proper retraining or adaptation. For the IFCB, the model was trained for instruments that are triggering on PMT-B, so particles and cells with no or limited chlorophyll may not be well represented here.

Bias, Risks, and Limitations

The model was trained on IFCB images collected at UCSC/MBARI and mostly in Monterey Bay, CA, USA or San Francisco Bay, CA, USA and may not generalize well to images from other instruments or regions. Users should validate model predictions with domain experts when possible.

How to Get Started with the Model

Install the transformers library and load the model as shown in the example above. For best results, use images preprocessed similarly to the IFCB dataset (see above).

Training Details

Training Data

The model was trained on approximately 558,000 labeled IFCB images representing 83 classes.

Training Procedure

Preprocessing: Images were resized and normalized consistent with ViT base requirements.

Evaluation

Testing Data, Factors & Metrics

The model was evaluated on a held-out test set of IFCB images.
Metrics include accuracy, precision, recall, and F1-score across phytoplankton classes.

Results

Label Name	precision	recall	f1-score	Eval #
Akashiwo	0.980405	0.984656	0.982526	2998
Alexandrium	0.972328	0.968642	0.970482	2902
Amylax_Gonyaulax_Protoceratium	0.987234	0.983051	0.985138	236
Asterionellopsis	0.982877	0.979522	0.981197	586
Asteromphalus	0.990488	0.988138	0.989311	843
Bad_setae	0.981581	0.969674	0.975591	1319
Centric	0.886133	0.848779	0.867054	2989
Ceratium_divaricatum	0.994825	0.97096	0.982748	792
Ceratium_furca	0.962202	0.966172	0.964183	1212
Ceratium_lineatum	0.975992	0.986287	0.981112	1896
Chaetoceros	0.944537	0.948	0.946265	3000
Ciliate_large	0.958333	0.974576	0.966387	118
Ciliate_large_2	0.959091	0.96789	0.96347	218
Ciliate_other_morpho_1	0.915578	0.918347	0.91696	992
Clusterflagellate_morpho_1	0.994539	0.982468	0.988467	1483
Clusterflagellate_morpho_2	0.992734	0.996354	0.99454	1097
Corethron	0.998889	0.99778	0.998334	901
Cryptophyte	0.951977	0.968391	0.960114	1740
Cylindrotheca	0.925259	0.969143	0.946693	1750
Detonula_Cerataulina_Lauderia	0.840866	0.880667	0.860306	3000
Detritus	0.971975	0.987915	0.97988	2317
Detritus_infection	0.996717	0.996308	0.996513	2438
Dictyocha	0.997705	0.995421	0.996562	2184
Dinoflagellate_cyst	1	1	1	17
Dinoflagellate_morpho_1	0.95098	0.984772	0.967581	394
Dinoflagellate_morpho_2	0.93253	0.940081	0.93629	2470
Dinophysis	0.986971	0.988581	0.987775	1226
Ditylum	0.994619	0.996406	0.995512	1113
Entomoneis	0.972626	0.978485	0.975547	1162
Eucampia	0.977153	0.926667	0.95124	3000
Euglenoid	0.972408	0.965145	0.968763	2410
Flagellate_morpho_1	0.966153	0.96132	0.963731	2999
Flagellate_morpho_2	0.942211	0.974026	0.957854	385
Flagellate_morpho_3	0.951259	0.969333	0.960211	3000
Flagellate_nano_1	0.956818	0.981352	0.96893	429
Flagellate_nano_2	0.988124	0.978824	0.983452	425
Fragilariopsis	0.900064	0.939667	0.919439	3000
Guinardia_Dactyliosolen	0.806818	0.913603	0.856897	544
Gymnodinium	0.830748	0.867452	0.848703	679
Gyrodinium	0.988604	0.991429	0.990014	1050
Gyrosigma	0.946237	0.946237	0.946237	93
Haptophyte_prymnesium	0.622642	0.673469	0.647059	49
Hemiaulus	0.903226	0.903226	0.903226	155
Hemiselmis	0.950862	0.974	0.962292	3000
Heterocapsa_long	0.958763	0.894231	0.925373	104
Heterocapsa_rotundata	0.964509	0.884211	0.922616	1045
Heterocapsa_triquetra	0.803571	0.656934	0.722892	137
Heterosigma_akashiwo	1	0.998477	0.999238	1313
Laboea	0.990521	0.987402	0.988959	635
Leptocylindrus	0.965558	0.949766	0.957597	856
Margalefidinium	0.973141	0.975378	0.974258	3046
Mesodinium	0.9583	0.962933	0.960611	2482
Nano_cluster	0.982955	0.997118	0.989986	347
Nano_p_white	0.982298	0.975951	0.979114	2786
Noctiluca	1	0.965517	0.982456	29
Odontella	1	1	1	30
Pennate	0.909332	0.864695	0.886452	3178
Pennate_Tropidoneis	0.837209	0.742268	0.786885	97
Pennate_Unknown	0.84127	0.828125	0.834646	64
Pennate_small	0.843373	0.864198	0.853659	405
Peridinium	0.968435	0.969086	0.96876	1488
Phaeocystis	0.994502	0.997931	0.996213	1450
Pleurosigma	0.991379	0.963149	0.97706	597
Polykrikos	0.997099	0.995174	0.996135	1036
Proboscia	0.992593	0.985294	0.98893	136
Prorocentrum_narrow	0.981952	0.981952	0.981952	2992
Prorocentrum_wide	0.988893	0.991463	0.990176	2694
Pseudo-nitzschia	0.956324	0.977674	0.966881	1075
Pyramimonas	1	0.982379	0.991111	227
Rhizosolenia	0.996008	0.984221	0.990079	507
Scrippsiella	0.960588	0.931015	0.94557	1754
Skeletonema	0.98632	0.993113	0.989705	1452
Spiky_pacman	0.961072	0.958908	0.959989	3553
Stombidinium_morpho_1	0.919847	0.909434	0.914611	265
Strombidinum_morpho_2	0.966399	0.940633	0.953342	2813
Thalassionema	0.989882	0.991554	0.990717	592
Thalassiosira	0.924272	0.931667	0.927955	3000
Tiarina	0.997843	0.996767	0.997305	928
Tontonia	0.954167	0.938525	0.946281	244
Torodinium	0.994792	0.990493	0.992638	1157
Tropidoneis	1	0.993569	0.996774	311
Vicicitus	0.943284	0.954683	0.948949	331
haptophyte_ucynA_host	1	0.998532	0.999265	2043
accuracy	0.958662	0.958662	0.958662	0.958662
macro avg	0.953973	0.951658	0.952527	111810
weighted avg	0.958948	0.958662	0.958652	111810

Technical Specifications

Model Architecture and Objective

Citation

If you use this model in your research, please cite:

APA:

Daniel, P. (2025). phytoViT_558k_Aug2025: Vision Transformer model for phytoplankton image classification. Retrieved from https://huggingface.co/phytoViT_558k_Aug2025

BibTeX:

@misc{daniel2025phytoViT,
  author = {Patrick Daniel},
  title = {phytoViT_558k_Aug2025: Vision Transformer model for phytoplankton image classification},
  year = {2025},
  howpublished = {\url{https://huggingface.co/phytoViT_558k_Aug2025}},
}

Model Card Authors

Patrick Daniel

Model Card Contact

pcdaniel@ucsc.edu

patcdaniel/UCSCPhytoViT83

Model Card for phytoViT_558k_Aug2025

Model Details

Model Description