Minisymposium Presentation

Foundation models have demonstrated great success in the field of natural language processing (NLP) and for other vision-based tasks (e.g., DALL-E). With the rise of data-driven, global weather forecast models, researchers have begun to create foundation models for the Earth System. There several foundation models in development (e.g., ClimaX) to allow for rapid fine-tuning to specific tasks such as weather forecasting or climate. However, several open questions remain on how well this foundation model approach will work with such a complex and diverse set of tasks typically needed for weather and climate. Here, we evaluate the ability for ClimaX to perform downstream tasks not seen during the pre-training phase. Specifically, we look at two tasks using ClimaX fine-tuned on ERA5 to 1) perform data assimilation using real, in-situ observations of the atmosphere and 2) replace the output layer of the foundation model with one that is parameterized by a Gaussian to perform uncertainty quantification (UQ). We also use the lessons learned from these experiments to develop a state-of-the-art weather forecasting model called Stormer. Stormer is a simple transformer-based model that achieves state-of-the-art performance on weather forecasting with minimal changes to the standard transformer backbone.