Minisymposium Presentation

A semi-implicit barotropic mode solver for the Model for Prediction Across Scales Ocean (MPAS-Ocean), an ocean component of the Energy Exascale Earth System Model (E3SM), has been ported on GPU using OpenACC directives. Since the semi-implicit solver in MPAS-O consists of a linear iterative solver and a preconditioner that requires linear algebra operations, we introduced the Matrix Algebra on GPU and Multicore Architecture (MAGMA) and CUBLAS which are collections of linear algebra libraries for heterogeneous architectures. We applied several methodologies such as algorithmic changes of the iterative solver, refactorization of loops, and the GPU-aware Message Passing Interface for the global all-to-all node communications to obtain optimized GPU performance. For runtime of main solver iterations including data staging, we achieved 5.4x (1.4x) speedup on 20 (100) Summit nodes. We will also show the GPU-accelerated solver performance using Cray LibSci_ACC supporting AMD MI250X GPU on Frontier. We will briefly discuss the recent update to MPAS-Ocean that changed the baroclinic time stepping method from the forward-backward to the second-order Adams Bashforth and its impact on the computational efficiency and model accuracy. This research is still underway, so methodologies may be further improved for better computational performance on GPUs.