TerraMind is the first multimodal any-to-any generative foundation model for Earth Observation jointly developed by IBM, ESA, and Forschungszentrum Jülich.
The model is pre-trained using FSQ-VAE tokens as targets. This tokenizer encodes and decodes land-use land-cover (LULC) maps for the TerraMind model.
The tokenizer uses FSQ with five dimensions and a codebook size of 4'375 tokens.
The model was pre-trained for 20 epochs on nine million LULC images from the TerraMesh dataset which are sourced from
ESRI.
The maps include nine classes and a 10th no-data class: No data, water, trees, flooded vegetation, crops, built area, bare ground, snow/ice, clouds, rangeland.
The tokenizer is fully integrated into the fine-tuning toolkit
TerraTorch.
You can initialize the pre-trained tokenizer with:
1from terratorch.registry import FULL_MODEL_REGISTRY
2model = FULL_MODEL_REGISTRY.build('terramind_v1_tokenizer_lulc', pretrained=True)
Once the model is build, it can be used to encode image and decode tokens.
1# Encode image
2_, _, tokens = model.encode(lulc_tensor)
3# Decode tokens
4reconstruction = model.decode_tokens(tokens)
5# Encode & decode
6reconstruction = model(lulc_tensor)
This tokenizer is automatically loaded with TerraMind generation models like
terramind_v1_base_generate, see
here for details.
We provide example code for the tokenizer at
https://github.com/IBM/terramind.
If you have feedback or any questions, please start a discussion in this HF repository or submitting an issue to
TerraMind on GitHub.
If you use TerraMind in your research, please cite our
TerraMind paper.
1@article{jakubik2025terramind,
2 title={TerraMind: Large-Scale Generative Multimodality for Earth Observation},
3 author={Jakubik, Johannes and Yang, Felix and Blumenstiel, Benedikt and Scheurer, Erik and Sedona, Rocco and Maurogiovanni, Stefano and Bosmans, Jente and Dionelis, Nikolaos and Marsocci, Valerio and Kopp, Niklas and others},
4 journal={IEEE/CVF International Conference on Computer Vision (ICCV)},
5 year={2025}
6}
