.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
.png)
Minisymposium Presentation
Advanced HPC Workflows for Urgent and Interactive Computing Using Julia
Presenter
Johannes Blaschke is a Computer Systems Engineer at the National Energy Research Scientific Computing Center (NERSC), specializing in workflow and application performance on HPC systems. He leads the NERSC Science Acceleration Program, and is helping design the next generation of supercomputers at NERSC. Johannes is passionate about urgent and interactive computing on supercomputers and emergent, highly heterogenous hardware. Before coming to NERSC, Johannes worked as a postdoc on developing fluid-structure interaction codes at the Technical University of Berlin, and at Lawrence Berkeley National Laboratory.
Description
Modern data-driven discovery algorithms and workflows require the tight interpretation of Simulation, Data Analysis, and AI. This means that all too often modern workflows fail to mesh well with HPC environments which are optimized for isolated applications over integrated workflows; and high utilization over fast feedback.
An example of this is real-time data analysis for experiment steering: time at large scientific instruments (such as particle accelerators, electron microscopes, or telescopes) is a scarce resource. Yet modern instruments often produce data at a rate that outpaces local computing resources. Therefore, scientists are turning to live data processing at HPC centers in order to gain the necessary insight to effectively steer their experiments.
In this talk, we demonstrate how Julia’s unique language features make it a natural language for developing tightly integrated Simulation + Analysis + AI workflows. We will also show an example of a workflow that flexibly grows its pool of compute nodes on an HPC system, thereby overcoming the constraints of the resource scheduler.