The FLAIR models are a collection of semantic segmentation models initially developed to classify land cover on very high resolution aerial images (more specifically the French BD ORTHO® product). The distributed pre-trained models differ in their :
dataset for training : FLAIR dataset or the increased version of this dataset FLAIR-INC (x 3.5 patches). Only the FLAIR dataset is open at the moment.
hardware: HPC/AI resources provided by GENCI-IDRIS
License: Etalab 2.0
Uses
Although the model can be applied to other type of very high spatial earth observation images, it was initially developed to tackle the problem of classifying aerial images acquired on the French Territory.
The product called (BD ORTHO®) has its own spatial and radiometric specifications. The model is not intended to be generic to other type of very high spatial resolution images but specific to BD ORTHO images.
Consequently, the model’s prediction would improve if the user images are similar to the original ones.
Radiometry of input images :
The BD ORTHO input images are distributed in 8-bit encoding format per channel. When traning the model, input normalization was performed (see section Training Details).
It is recommended that the user apply the same type of input normalization while inferring the model.
Multi-domain model :
The FLAIR-INC dataset that was used for training is composed of 75 radiometric domains. In the case of aerial images, domain shifts are frequent and are mainly due to : the date of acquisition of the aerial survey (from april to november), the spatial domain (equivalent to a french department administrative division) and downstream radiometric processing.
By construction (sampling 75 domains) the model is robust to these shifts, and can be applied to any images of the (BD ORTHO® product).
Land Cover classes of prediction :
The orginial class nomenclature of the FLAIR Dataset encompasses 19 classes (See the FLAIR dataset page for details).
However 3 classes corresponding to uncertain labelisation (Mixed (16), Ligneous (17) and Other (19)) and 1 class with very poor labelling (Clear cut (15)) were desactivated during training.
As a result, the logits produced by the model are of size 19x1, but classes n° 15, 16, 17 and 19 should appear at 0 in the logits and should not be present in the final argmax product.
Bias, Risks, Limitations and Recommendations
Using the model on input images with other spatial resolution :
The FLAIR-INC_rgb_15cl_resnet34-fpn model was trained with fixed scale conditions. All patches used for training are derived from aerial images with 0.2 meters spatial resolution. Only flip and rotate augmentations were performed during the training process.
No data augmentation method concerning scale change was used during training. The user should pay attention that generalization issues can occur while applying this model to images that have different spatial resolutions.
Using the model for other remote sensing sensors :
The FLAIR-INC_rgb_15cl_resnet34-fpn model was trained with aerial images of the (BD ORTHO® product) that encopass very specific radiometric image processing.
Using the model on other type of aerial images or satellite images may imply the use of transfer learning or domain adaptation techniques.
Using the model on other spatial areas :
The FLAIR-INC_rgb_15cl_resnet34-fpn model was trained on patches reprensenting the French Metropolitan territory.
The user should be aware that applying the model to other type of landscapes may imply a drop in model metrics.
218 400 patches of 512 x 512 pixels were used to train the FLAIR-INC_rgb_15cl_resnet34-fpn model.
The train/validation split was performed patchwise to obtain a 80% / 20% distribution between train and validation.
Annotation was performed at the zone level (~100 patches per zone). Spatial independancy between patches is guaranted as patches from the same zone were assigned to the same set (TRAIN or VALIDATION).
The following number of patches were used for train and validation :
| TRAIN set | 174 700 patches |
| VALIDATION set | 43 700 patchs |
Training Procedure
Preprocessing
For traning the model, input normalization was performed to center-reduce (a mean=0 and a standard deviation = 1, channel wise) the dataset.
We used the statistics of TRAIN+VALIDATION for input normalization. It is recommended that the user apply the same type of input normalization.
The FLAIR-INC_rgb_15cl_resnet34-fpn model was trained on a HPC/AI resources provided by GENCI-IDRIS (Grant 2022-A0131013803).
16 V100 GPUs were used ( 4 nodes, 4 GPUS per node). With this configuration the approximate learning time is 6 minutes per epoch.
FLAIR-INC_rgb_15cl_resnet34-fpn was obtained for num_epoch=35 with corresponding val_loss=0.60.
TRAIN loss
TRAIN loss
VALIDATION loss
VALIDATION loss
Evaluation
Testing Data, Factors & Metrics
Testing Data
The evaluation was performed on a TEST set of 31 750 patches that are independant from the TRAIN and VALIDATION patches. They represent 15 spatio-temporal domains.
The TEST set corresponds to the reunion of the TEST set of scientific challenges FLAIR#1 and FLAIR#2. See the FLAIR challenge page for more details.
The choice of a separate TEST set instead of cross validation was made to be coherent with the FLAIR challenges.
However the metrics for the Challenge were calculated on 12 classes and the TEST set acordingly.
As a result the Snow class is absent from the TEST set.
Metrics
With the evaluation protocol, the FLAIR-INC_rgb_15cl_resnet34-fpn have been evaluated to OA= 74.712% and mIoU=59.284%.
The snow class is discarded from the average metrics.
The following table give the class-wise metrics :
Classes
IoU (%)
Fscore (%)
Precision (%)
Recall (%)
building
76.291
86.551
86.076
87.032
pervious_surface
53.936
70.076
71.199
68.988
impervious_surface
70.233
82.514
81.715
83.329
bare_soil
58.884
74.122
74.358
73.887
water
83.083
90.760
88.987
92.605
coniferous
57.134
72.720
75.998
69.714
deciduous
69.199
81.796
78.513
85.365
brushwood
29.852
45.978
57.297
38.393
vineyard
77.437
87.284
82.969
92.072
herbaceous
48.452
65.277
71.663
59.935
agricultural_land
56.813
72.460
64.846
82.100
plowed_land
41.103
58.260
59.095
57.448
swimming_pool
42.840
59.984
71.155
51.844
snow
0.000
0.000
0.000
0.000
greenhouse
64.718
78.580
73.736
84.106
average
59.284
73.312
74.115
73.344
The following illustration gives the resulting confusion matrix :
Top : normalised acording to columns, columns sum at 100% and the precision is on the diagonal of the matrix
Bottom : normalised acording to rows, rows sum at 100% and the recall is on the diagonal of the matrix
Normalized Confusion Matrix (precision)
drawing
Normalized Confusion Matrix (recall)
drawing
Results
Samples of results
Citation
BibTeX:
@inproceedings{ign-flair,
title={FLAIR: a Country-Scale Land Cover Semantic Segmentation Dataset From Multi-Source Optical Imagery},
author={Anatol Garioud and Nicolas Gonthier and Loic Landrieu and Apolline De Wit and Marion Valette and Marc Poupée and Sébastien Giordano and Boris Wattrelos},
year={2023},
booktitle={Advances in Neural Information Processing Systems (NeurIPS) 2023},
doi={https://doi.org/10.48550/arXiv.2310.13336},
}
APA:
Anatol Garioud, Nicolas Gonthier, Loic Landrieu, Apolline De Wit, Marion Valette, Marc Poupée, Sébastien Giordano and Boris Wattrelos. 2023.
FLAIR: a Country-Scale Land Cover Semantic Segmentation Dataset From Multi-Source Optical Imagery. (2023).
In proceedings of Advances in Neural Information Processing Systems (NeurIPS) 2023.
DOI: https://doi.org/10.48550/arXiv.2310.13336
