Heterogeneity and Observability: A TAU Perspective from Then, Now, and Beyond

Short abstract:

Throughout the evolution of HPC, the ability to create more scalable and sophisticated applications has been intimately coupled with advances in hardware capabilities, runtime software, and programming environments. In general, each generation of HPC is more complex to operate, program, and use than before.  Moreover, the increased complexity is reflected in a greater performance complexity that needs to be understood and managed in order to operate these machines at the high end of their performance range.

It might be said that HPC innovation is prone to disruption, whereby that new features can force (radical) shift in parallelism approach.  In recent years, HPC has been addressing the rise of heterogeneity, especially where post-Moore computer architecture innovations are resulting in a diversity of processors, accelerators, memory, and interconnects.  While there are certainly opportunities to build more powerful HPC systems based on these technological advances, the operational complexity of the heterogeneous system as a whole makes it more challenging.

Here we argue that heterogeneity (as an abstract concept of HPC diversity and change) is closely associated with observability (as an abstract concept of performance).  More directly, as HPC complexity increases, so is the need for more/better performance methods, techniques, and tools.  Future HPC systems will be more complex (not simpler), necessitating greater support for observability (not less).  The conclusion of this argument is that future HPC systems and the applications that run on them will not be able to deliver their full potential unless observability solutions are more complete, integrated, and smarter.