  Identifying phase transitions and classifying phases of matter is central to
understanding the properties and behavior of a broad range of material systems.
In recent years, machine-learning (ML) techniques have been successfully
applied to perform such tasks in a data-driven manner. However, the success of
this approach notwithstanding, we still lack a clear understanding of ML
methods for detecting phase transitions, particularly of those that utilize
neural networks (NNs). In this work, we derive analytical expressions for the
optimal output of three widely used NN-based methods for detecting phase
transitions. These optimal predictions correspond to the results obtained in
the limit of high model capacity. Therefore, in practice they can, for example,
be recovered using sufficiently large, well-trained NNs. The inner workings of
the considered methods are revealed through the explicit dependence of the
optimal output on the input data. By evaluating the analytical expressions, we
can identify phase transitions directly from experimentally accessible data
without training NNs, which makes this procedure favorable in terms of
computation time. Our theoretical results are supported by extensive numerical
simulations covering, e.g., topological, quantum, and many-body localization
phase transitions. We expect similar analyses to provide a deeper understanding
of other classification tasks in condensed matter physics.
