  Burst-Buffering is a promising storage solution that introduces an
intermediate highthroughput storage buffer layer to mitigate the I/O bottleneck
problem that the current High-Performance Computing (HPC) platforms suffer. The
existing Markov-Chain based probabilistic I/O scheduling utilizes the load
state of Burst-Buffers and the periodical characteristics of applications to
reduce I/O congestion due to the limited capacity of Burst-Buffers. However,
this probabilistic approach requires consistent I/O characteristics of
applications, including similar I/O duration and long application length, in
order to obtain an accurate I/O load estimation. These consistency conditions
do not often hold in realistic situations.
  In this paper, we propose a generic framework of dynamic probabilistic I/O
scheduling based on application clustering (DPSAC) to make applications meet
the consistency requirements. According to the I/O phrase length of each
application, our scheme first deploys a one-dimensional K-means clustering
algorithm to cluster the applications into clusters. Next, it calculates the
expected workload of each cluster through the probabilistic model of
applications and then partitions the Burst-Buffers proportionally. Then, to
handle dynamic changes (join and exit) of applications, it updates the clusters
based on a heuristic strategy. Finally, it applies the probabilistic I/O
scheduling, which is based on the distribution of application workload and the
state of Burst-Buffers, to schedule I/O for all the concurrent applications to
mitigate I/O congestion. The simulation results on synthetic data show that our
DPSAC is effective and efficient.
