  Workload characterization is an integral part of performance analysis of high
performance computing (HPC) systems. An understanding of workload properties
sheds light on resource utilization and can be used to inform performance
optimization both at the software and system configuration levels. It can
provide information on how computational science usage modalities are changing
that could potentially aid holistic capacity planning for the wider HPC
ecosystem. Here, we report on the results of a detailed workload analysis of
the portfolio of supercomputers comprising the NSF Innovative HPC program in
order to characterize its past and current workload and look for trends to
understand the nature of how the broad portfolio of computational science
research is being supported and how it is changing over time. The workload
analysis also sought to illustrate a wide variety of usage patterns and
performance requirements for jobs running on these systems. File system
performance, memory utilization and the types of parallelism employed by users
(MPI, threads, etc) were also studied for all systems for which job level
performance data was available.
