AIFS produces highly skilled forecasts for upper-air variables, surface weather parameters and
tropical cyclone tracks. AIFS ENS is run four times daily alongside ECMWF’s physics-based NWP model and forecasts
are available to the public under ECMWF’s open data policy.
Note that due to the non-determinism of GPUs, users will be unable to exactly reproduce an official AIFS ENS forecast
when running AIFS ENS themselves.
Note
Regarding precipitation units, it's worth noting that AIFS model was trained on \(m^{3}/m^{2}\) and will therefore produce precip in that units.
If one wants to retrieve precipitation from Open data, the units will be \(mm\).
AIFS is based on a graph neural network (GNN) encoder and decoder, and a sliding window transformer processor,
and is trained on ECMWF’s ERA5 re-analysis and ECMWF’s operational numerical weather prediction (NWP) analyses.
Encoder graph
Decoder graph
It has a flexible and modular design and supports several levels of parallelism to enable training on
high resolution input data. AIFS forecast skill is assessed by comparing its forecasts to NWP analyses
and direct observational data.
AIFS ENS was trained to minimise the Continuous Ranked Probability Score (CRPS) calculated over
a small ensemble.
Repository:Anemoi is an open-source framework for
creating machine learning (ML) weather forecasting systems, co-developed by ECMWF and a range of national meteorological
services across Europe.
🚨 Noteaifs-ens-1.0 requires 38GB of GPU memory by default, it is possible to reduce this by setting ANEMOI_INFERENCE_NUM_CHUNKS to a multiple of 2.
For example setting it to 16 reduces the memory requirements to 24GB at the cost of throughput. Therefore issues may arise with lower end GPUs.
To generate a new forecast using AIFS ENS, you can use anemoi-inference. In the example notebook, a
step-by-step workflow is specified to run AIFS ENS using the HuggingFace model:
Add the single levels fields and pressure levels fields
Convert geopotential height into geopotential
Create the initial state
Load the Model and Run the Forecast
Download the Model's Checkpoint from Hugging Face
Create a runner
Run the forecast using anemoi-inference
Inspect the generated forecast
Plot a field
🚨 Note we train AIFS using flash_attention.
The use of the 'Flash Attention' package also imposes certain requirements in terms of software and hardware, which can be found under
Installation and Features on the GitHub repo.
🚨 Note the aifs_ens_v1.0.ckpt checkpoint just contains the model’s weights. It does not contain any information about the optimizer states,
lr-scheduler states, etc.
Citation
If you use this model in your work, please cite it as follows:
BibTeX:
@misc{lang2024aifscrps,
title={AIFS-CRPS: Ensemble forecasting using a model trained with a loss function based on the Continuous Ranked Probability Score},
author={Simon Lang and Mihai Alexe and Mariana C. A. Clare and Christopher Roberts and Rilwan Adewoyin and Zied Ben Bouallègue and Matthew Chantry and Jesper Dramsch and Peter D. Dueben and Sara Hahner and Pedro Maciel and Ana Prieto-Nemesio and Cathal O'Brien and Florian Pinault and Jan Polster and Baudouin Raoult and Steffen Tietsche and Martin Leutbecher},
year={2024},
eprint={2412.15832},
archivePrefix={arXiv},
primaryClass={physics.ao-ph},
url={https://arxiv.org/abs/2412.15832},
}
APA:
Lang, S., Alexe, M., Clare, M. C., Roberts, C., Adewoyin, R., Bouallègue, Z. B., ... & Leutbecher, M. (2024). AIFS-CRPS: Ensemble forecasting using a model trained with a loss function based on the Continuous Ranked Probability Score. arXiv preprint arXiv:2412.15832.
