  Evaluating classifications is crucial in statistics and machine learning, as
it influences decision-making across various fields, such as patient prognosis
and therapy in critical conditions. The Matthews correlation coefficient (MCC)
is recognized as a performance metric with high reliability, offering a
balanced measurement even in the presence of class imbalances. Despite its
importance, there remains a notable lack of comprehensive research on the
statistical inference of MCC. This deficiency often leads to studies merely
validating and comparing MCC point estimates, a practice that, while common,
overlooks the statistical significance and reliability of results. Addressing
this research gap, our paper introduces and evaluates several methods to
construct asymptotic confidence intervals for the single MCC and the
differences between MCCs in paired designs. Through simulations across various
scenarios, we evaluate the finite-sample behavior of these methods and compare
their performances. Furthermore, through real data analysis, we illustrate the
potential utility of our findings in comparing binary classifiers, highlighting
the possible contributions of our research in this field.
