The granite-geospatial-biomass model is a fine-tuned geospatial foundation model for predicting the total above ground biomass (i.e., living and dead plant material on the Earth's surface) using optical satellite imagery.
Above ground biomass is an important component of the carbon cycle and is crucial for estimating crop yields, monitoring forest timber production, and quantifying the carbon sequestered by nature-based actions.
The model predicts above ground biomass from the Harmonized Landsat and Sentinel-2 (HLS) L30 optical satellite imagery and is fine-tuned using training labels from the
Global Ecosystem Dynamics Investigation (GEDI) L4A. Uniquely, the model has been fine-tuned using HLS and GEDI data collected from 15 biomes across the globe.
Please see [Model Description](Model Description) below for more details.
These notebooks guide the user through three experiments:
Zero-shot for all biomes
Zero-shot for a single biome
Few-shot for a single biome
Model Description
The granite-geospatial-biomass model is a geospatial foundation model that has been fine-tuned using HLS and GEDI data to perform regression.
The base foundation model from which the granite-geospatial-biomass model is fine-tuned is similar to that described in this paper,
with the exception that the backbone is a Swin-B transformer. We opted for the Swin-B backbone instead of the ViT in the original paper because the Swin-B provides the following advantages:
a smaller starting patch size which provides a higher effective resolution
windowed attention which provides better computational efficiency
hierarchical merging which provides a useful inductive bias
The base foundation model was pretrained using SimMIM, a self-supervised learning strategy based on masking large parts of the input (HLS2) data which are then reconstructed by the model. A small decoder composed of a single convolutional layer and a Pixel Shuffle module was added to the Swin-B backbone for the (pretraining) reconstruction task.
For fine-tuning, we replaced the small decoder with a UPerNet adapted for pixel-wise regression. We opted for the UPerNet because it provides fusion between transformer blocks, a similar intuition to the Unet which is consistently considered state-of-the-art for regression tasks with earth observation data. As the standard UPerNet implementation using the Swin-B backbone predicts a final feature map 4x smaller than the input, we appended two Pixel Shuffle layers to learn the upscaling. More details on the fine-tuned model can be found in this paper.
Model Releases (along with the branch name where the models are stored):
For training and testing, the model requires a cloud-free snapshot of an area where all pixels are representative of the spectral bands for that location. The approach we used to create the cloud free images was to acquire HLS data during the leaf-on season for each hemisphere, analyze the timeseries, and select pixels that are not contaminated with clouds. We compute the mean value of each cloud-free pixel during the leaf-on season for each spectral band which is then assembled into a composite image representative for that area. The corresponding GEDI L4A biomass data obtained made during the same leaf-on season are interpolated to the HLS grid (CRS:4326) such that the measured biomass points are aligned with HLS data. GEDI data is spatially and temporaly sparse so pixels with no corresponding GEDI measurement are filled with a no data value.
Citation [optional]
Kindly cite the following paper, if you intend to use our model or its associated architectures/approaches in your
work
BibTeX:
@misc{muszynski2024finetuninggeospatialfoundationmodels,
title={Fine-tuning of Geospatial Foundation Models for Aboveground Biomass Estimation},
author={Michal Muszynski and Levente Klein and Ademir Ferreira da Silva and Anjani Prasad Atluri and Carlos Gomes and Daniela Szwarcman and Gurkanwar Singh and Kewen Gu and Maciel Zortea and Naomi Simumba and Paolo Fraccaro and Shraddha Singh and Steve Meliksetian and Campbell Watson and Daiki Kimura and Harini Srinivasan},
year={2024},
url={https://arxiv.org/abs/2406.19888},
}
APA:
Muszynski, M., Klein, L., da Silva, A. F., Atluri, A. P., Gomes, C., Szwarcman, D., Singh, G., Gu, K.,
Zortea, M., Simumba, N., Fraccaro, P., Singh, S., Meliksetian, S., Watson, C., Kimura, D., & Srinivasan, H. (2024).
Fine-tuning of geospatial foundation models for aboveground biomass estimation. arXiv. https://arxiv.org/abs/2406.19888
Model Card Authors
Julian Kuehnert, Levente Klein, Catherine Wanjiru, Carlos Gomes and Campbell Watson
IBM Public Repository Disclosure:
All content in this repository including code has been provided by IBM under the associated
open source software license and IBM is under no obligation to provide enhancements,
updates, or support. IBM developers produced this code as an
open source project (not as an IBM product), and IBM makes no assertions as to
the level of quality nor security, and will not be maintaining this code going forward.
