  Estimation of the Fisher Information Metric (FIM-estimation) is an important
task that arises in unsupervised learning of phase transitions, a problem
proposed by physicists. This work completes the definition of the task by
defining rigorous evaluation metrics distMSE, distMSEPS, and distRE and
introduces ClassiFIM, a novel machine learning method designed to solve the
FIM-estimation task. Unlike existing methods for unsupervised learning of phase
transitions, ClassiFIM directly estimates a well-defined quantity (the FIM),
allowing it to be rigorously compared to any present and future other methods
that estimate the same. ClassiFIM transforms a dataset for the FIM-estimation
task into a dataset for an auxiliary binary classification task and involves
selecting and training a model for the latter. We prove that the output of
ClassiFIM approaches the exact FIM in the limit of infinite dataset size and
under certain regularity conditions. We implement ClassiFIM on multiple
datasets, including datasets describing classical and quantum phase
transitions, and find that it achieves a good ground truth approximation with
modest computational resources. Furthermore, we independently implement two
alternative state-of-the-art methods for unsupervised estimation of phase
transition locations on the same datasets and find that ClassiFIM predicts such
locations at least as well as these other methods. To emphasize the generality
of our method, we also propose and generate the MNIST-CNN dataset, which
consists of the output of CNNs trained on MNIST for different hyperparameter
choices. Using ClassiFIM on this dataset suggests there is a phase transition
in the distribution of image-prediction pairs for CNNs trained on MNIST,
demonstrating the broad scope of FIM-estimation beyond physics.
