  Modern branch predictors predict the vast majority of conditional branch
instructions with near-perfect accuracy, allowing superscalar, out-of-order
processors to maximize speculative efficiency and thus performance. However,
this impressive overall effectiveness belies a substantial missed opportunity
in single-threaded instructions per cycle (IPC). For example, we show that
correcting the mispredictions made by the state-of-the-art TAGE-SC-L branch
predictor on SPECint 2017 would improve IPC by margins similar to an advance in
process technology node.
  In this work, we measure and characterize these mispredictions. We find that
they categorically arise from either (1) a small number of systematically
hard-to-predict (H2P) branches; or (2) rare branches with low dynamic execution
counts. Using data from SPECint 2017 and additional large code footprint
applications, we quantify the occurrence and IPC impact of these two
categories. We then demonstrate that increasing the resources afforded to
existing branch predictors does not alone address the root causes of most
mispredictions. This leads us to reexamine basic assumptions in branch
prediction and to propose new research directions that, for example, deploy
machine learning to improve pattern matching for H2Ps, and use on-chip phase
learning to track long-term statistics for rare branches.
