  Efficient and automated classification of phases from minimally processed
data is one goal of machine learning in condensed matter and statistical
physics. Supervised algorithms trained on raw samples of microstates can
successfully detect conventional phase transitions via learning a bulk feature
such as an order parameter. In this paper, we investigate whether neural
networks can learn to classify phases based on topological defects. We address
this question on the two-dimensional classical XY model which exhibits a
Kosterlitz-Thouless transition. We find significant feature engineering of the
raw spin states is required to convincingly claim that features of the vortex
configurations are responsible for learning the transition temperature. We
further show a single-layer network does not correctly classify the phases of
the XY model, while a convolutional network easily performs classification by
learning the global magnetization. Finally, we design a deep network capable of
learning vortices without feature engineering. We demonstrate the detection of
vortices does not necessarily result in the best classification accuracy,
especially for lattices of less than approximately 1000 spins. For larger
systems, it remains a difficult task to learn vortices.
