  Although high-performance computing (HPC) systems have been scaled to meet
the exponentially-growing demand for scientific computing, HPC performance
variability remains a major challenge and has become a critical research topic
in computer science. Statistically, performance variability can be
characterized by a distribution. Predicting performance variability is a
critical step in HPC performance variability management and is nontrivial
because one needs to predict a distribution function based on system factors.
In this paper, we propose a new framework to predict performance distributions.
The proposed model is a modified Gaussian process that can predict the
distribution function of the input/output (I/O) throughput under a specific HPC
system configuration. We also impose a monotonic constraint so that the
predicted function is nondecreasing, which is a property of the cumulative
distribution function. Additionally, the proposed model can incorporate both
quantitative and qualitative input variables. We evaluate the performance of
the proposed method by using the IOzone variability data based on various
prediction tasks. Results show that the proposed method can generate accurate
predictions, and outperform existing methods. We also show how the predicted
functional output can be used to generate predictions for a scalar summary of
the performance distribution, such as the mean, standard deviation, and
quantiles. Our methods can be further used as a surrogate model for HPC system
variability monitoring and optimization.
