Minisymposium
MS3A - Towards km-Scale Weather and Climate Simulations on (pre-)Exascale HPC Systems
Replay
Session Chair
Description
Weather and climate is placed as one of the prime application domains for the incipient era of Exascale computing. The unprecedented amount of resources allows for the first time to perform simulations at the km-scale for climate time scales but poses huge challenges for developers of Earth system models, which are often comprised of large monolithic Fortran code bases. It incurs a need for large-scale refactoring, adaptation to new programming models and a transition to a scalable and sustainable development process that allows to continuously adapt to the rapidly developing portfolio of new hardware and software stacks.
Presentations
EarthWorks is a US NSF-funded project that is led through Colorado State University with a partnership through NCAR which aims to run a fully coupled Earth system model at 3.75 km resolution, global storm-resolving resolutions. The EarthWorks’ model configuration leverages the infrastructure of the Community Earth System Model (CESM) to couple the CAM-MPAS model with the MPAS Ocean, MPAS Sea Ice, and the Community Land Model (CLM) models.Our fully coupled model configuration has been successfully tested in multi-year simulations down to the 15 km resolution on CPU’s, with shorter functional tests being completed at our target resolution of 3.75 km. In order to achieve our performance targets, we need to create a GPU resident version of our model. This presentation will focus on the refactoring work that is well underway and the acceleration that has been obtained thus far. We will also discuss the software engineering challenges that this project has faced and how they have been overcome.
Destination Earth (DestinE) is an ambitious initiative of the European Union to create digital twins of our planet. As extreme weather becomes increasingly frequent and changes in climate more pronounced, there is an urgent need to forecast these events with even greater accuracy, to predict their impact on the environment, life and property. This talk will present the work led by ECMWF in DestinE in building the Digital Twin Engine and developing the Weather Extremes and Climate Change Adaption digital twins. We show the work done on developing and deploying the Digital Twin Engine on EuroHPC systems such as LUMI and Leonardo and share our experiences and challenges to date in using these systems for running kilometre scale weather and climate digital twins.
The Icosahedral Nonhydrostatic (ICON) weather and climate model is a modelling framework for numerical weather prediction and climate simulations. ICON is implemented mostly in Fortran 2008 with the GPU version based mainly on OpenACC. ICON is used on a large variety of hardware, ranging from classical CPU cluster to vector architecture and different GPU systems.
An ICON model configuration developed for km-scale climate simulations is used as a scientific prototype for the digital twin of the Earth for climate adaptation within the Destination Earth program of the European Comission. Here we focus on our effort to run these coupled ICON configurations at km-scale on LUMI, a HPE Cray EX system with a GPU partition based on AMD MI250x’s.
We present the model configuration, performance results and scalability of the simulation system on Lumi and compare it with results on other GPU and CPU based systems.
Within EXCLAIM (EXtreme scale Computing and data platform for cLoud-resolving weAther and clImate Modeling), we are developing a performance-portable weather and climate model that resides within a Python framework. The model is based on the German global non-hydrostatic modeling system ICON. Significant progress has been made in the last two years. Model components (i.e., dynamical core, tracer advection, and cloud microphysics) responsible for the model physics have been rewritten using an embedded DSL called GT4Py. The Python driver code that combines these physical components with the model infrastructure for a successful simulation is demonstrated to perform satisfactorily for a standard dynamical core test. Work is ongoing to expand the Python framework by including other model components. While there were several challenges along the way, the challenge that remains open is the missing connection with the conventional model developers which, by and large, is acting as an observer.