DinoVd'eau is a fine-tuned version of microsoft/resnet-50 on the multilabel_complete_dataset dataset. It achieves the following results on the evaluation set:

Loss: 0.1518
F1 Micro: 0.7545
F1 Macro: 0.6309
Roc Auc: 0.8276
Accuracy: 0.4069
Learning Rate: 1e-05

Model description

DinoVd'eau is a model built on top of dinov2 model for underwater multilabel image classification.The classification head is a combination of linear, ReLU, batch normalization, and dropout layers.

Developed by: lombardata, credits to César Leblanc

Intended uses & limitations

You can use the raw model for classify diverse marine species, encompassing coral morphotypes classes taken from the Global Coral Reef Monitoring Network (GCRMN), habitats classes and seagrass species.

Training and evaluation data

Details on the number of images for each class are given in the following table:

	train	val	test	Total
Acropore_branched	804	202	200	1206
Acropore_digitised	465	108	101	674
Acropore_tabular	964	276	267	1507
Algae_assembly	2172	692	698	3562
Algae_limestone	1327	439	441	2207
Algae_sodding	2079	676	671	3426
Dead_coral	1126	358	355	1839
Fish	874	243	242	1359
Human_object	407	135	136	678
Living_coral	1765	580	571	2916
Millepore	350	119	102	571
No_acropore_encrusting	411	142	129	682
No_acropore_foliaceous	212	34	39	285
No_acropore_massive	921	317	310	1548
No_acropore_sub_massive	1205	362	363	1930
Rock	3736	1218	1217	6171
Sand	3594	1202	1194	5990
Scrap	2121	724	741	3586
Sea_cucumber	781	254	265	1300
Sea_urchins	189	60	72	321
Sponge	226	75	88	389
Syringodium_isoetifolium	1171	386	392	1949
Thalassodendron_ciliatum	783	261	260	1304
Useless	587	195	195	977

Training procedure

Data Augmentation

Data were augmented using the following transformations :

training transformations : Sequential( (0): PreProcess() (1): Resize(output_size=(518, 518), p=1.0, p_batch=1.0, same_on_batch=True, size=(518, 518), side=short, resample=bilinear, align_corners=True, antialias=False) (2): RandomHorizontalFlip(p=0.25, p_batch=1.0, same_on_batch=False) (3): RandomVerticalFlip(p=0.25, p_batch=1.0, same_on_batch=False) (4): ColorJiggle(brightness=0.0, contrast=0.0, saturation=0.0, hue=0.0, p=0.25, p_batch=1.0, same_on_batch=False) (5): RandomPerspective(distortion_scale=0.5, p=0.25, p_batch=1.0, same_on_batch=False, align_corners=False, resample=bilinear) (6): Normalize(p=1.0, p_batch=1.0, same_on_batch=True, mean=tensor([0.4850, 0.4560, 0.4060]), std=tensor([0.2290, 0.2240, 0.2250])) )
validation transformations : Sequential( (0): PreProcess() (1): Resize(output_size=(518, 518), p=1.0, p_batch=1.0, same_on_batch=True, size=(518, 518), side=short, resample=bilinear, align_corners=True, antialias=False) (2): Normalize(p=1.0, p_batch=1.0, same_on_batch=True, mean=tensor([0.4850, 0.4560, 0.4060]), std=tensor([0.2290, 0.2240, 0.2250])) )

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: ReduceLROnPlateau with a patience of 5 epochs and a factor of 0.1
freeze_encoder: True
num_epochs: 93

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Micro	F1 Macro	Roc Auc	Accuracy	Rate
No log	1.0	274	0.2237	0.5839	0.2834	0.7176	0.1952	0.001
0.2683	2.0	548	0.1895	0.6773	0.4549	0.7743	0.3055	0.001
0.2683	3.0	822	0.1786	0.7021	0.5202	0.7911	0.3539	0.001
0.2058	4.0	1096	0.1715	0.7198	0.5666	0.8058	0.3667	0.001
0.2058	5.0	1370	0.1662	0.7220	0.5718	0.8050	0.3768	0.001
0.1916	6.0	1644	0.1648	0.7155	0.5721	0.7980	0.3796	0.001
0.1916	7.0	1918	0.1618	0.7281	0.5973	0.8082	0.3810	0.001
0.1858	8.0	2192	0.1598	0.7375	0.6061	0.8166	0.3855	0.001
0.1858	9.0	2466	0.1599	0.7440	0.6209	0.8223	0.3911	0.001
0.1839	10.0	2740	0.1584	0.7382	0.6047	0.8173	0.3949	0.001
0.1815	11.0	3014	0.1569	0.7414	0.6068	0.8186	0.3960	0.001
0.1815	12.0	3288	0.1585	0.7257	0.5953	0.8043	0.3963	0.001
0.1807	13.0	3562	0.1581	0.7514	0.6286	0.8311	0.3967	0.001
0.1807	14.0	3836	0.1565	0.7453	0.6230	0.8224	0.4022	0.001
0.1795	15.0	4110	0.1549	0.7504	0.6253	0.8262	0.3991	0.001
0.1795	16.0	4384	0.1573	0.7446	0.6290	0.8214	0.3939	0.001
0.178	17.0	4658	0.1551	0.7519	0.6287	0.8281	0.4026	0.001
0.178	18.0	4932	0.1570	0.7430	0.6155	0.8203	0.3914	0.001
0.1764	19.0	5206	0.1558	0.7480	0.6287	0.8236	0.3991	0.001
0.1764	20.0	5480	0.1574	0.7403	0.6085	0.8164	0.4001	0.001
0.1775	21.0	5754	0.1561	0.7532	0.6246	0.8302	0.4029	0.001
0.177	22.0	6028	0.1545	0.7596	0.6431	0.8378	0.3974	0.0001
0.177	23.0	6302	0.1556	0.7472	0.6292	0.8233	0.4026	0.0001
0.1762	24.0	6576	0.1548	0.7528	0.6343	0.8283	0.3994	0.0001
0.1762	25.0	6850	0.1554	0.7468	0.6225	0.8222	0.3994	0.0001
0.1759	26.0	7124	0.1548	0.7529	0.6326	0.8297	0.3977	0.0001
0.1759	27.0	7398	0.1552	0.7516	0.6352	0.8282	0.3970	0.0001
0.1752	28.0	7672	0.1543	0.7523	0.6328	0.8277	0.4092	0.0001
0.1752	29.0	7946	0.1545	0.7506	0.6312	0.8265	0.4019	0.0001
0.1757	30.0	8220	0.1550	0.7554	0.6394	0.8340	0.4040	0.0001
0.1757	31.0	8494	0.1554	0.7512	0.6345	0.8279	0.4022	0.0001
0.1758	32.0	8768	0.1545	0.7513	0.6302	0.8275	0.4033	0.0001
0.1755	33.0	9042	0.1555	0.7456	0.6261	0.8223	0.3977	0.0001
0.1755	34.0	9316	0.1533	0.7515	0.6307	0.8260	0.4109	0.0001
0.1752	35.0	9590	0.1551	0.7506	0.6325	0.8261	0.4054	0.0001
0.1752	36.0	9864	0.1530	0.7539	0.6299	0.8287	0.4026	0.0001
0.1752	37.0	10138	0.1546	0.7464	0.6270	0.8223	0.4036	0.0001
0.1752	38.0	10412	0.1549	0.7539	0.6364	0.8314	0.3987	0.0001
0.1763	39.0	10686	0.1547	0.7579	0.6421	0.8361	0.3977	0.0001
0.1763	40.0	10960	0.1544	0.7539	0.6345	0.8302	0.4005	0.0001
0.176	41.0	11234	0.1557	0.7536	0.6347	0.8298	0.4015	0.0001
0.1758	42.0	11508	0.1540	0.7474	0.6277	0.8226	0.3960	0.0001
0.1758	43.0	11782	0.1548	0.7578	0.6384	0.8374	0.3970	1e-05
0.1764	44.0	12056	0.1543	0.7582	0.6398	0.8352	0.4012	1e-05
0.1764	45.0	12330	0.1544	0.7448	0.6206	0.8196	0.3991	1e-05
0.1746	46.0	12604	0.1546	0.7452	0.6223	0.8208	0.4050	1e-05

Framework versions

Transformers 4.36.2
Pytorch 2.1.0+cu118
Datasets 2.14.5
Tokenizers 0.15.0

lombardata/resnet-50-linearhead-2024_03_12-with_data_aug_batch-size32_epochs93_freeze

resnet-50-linearhead-2024_03_12-with_data_aug_batch-size32_epochs93_freeze