  Despite its centrality in the philosophy of cognitive science, there has been
little prior philosophical work engaging with the notion of representation in
contemporary NLP practice. This paper attempts to fill that lacuna: drawing on
ideas from cognitive science, I introduce a framework for evaluating the
representational claims made about components of neural NLP models, proposing
three criteria with which to evaluate whether a component of a model represents
a property and operationalising these criteria using probing classifiers, a
popular analysis technique in NLP (and deep learning more broadly).
  The project of operationalising a philosophically-informed notion of
representation should be of interest to both philosophers of science and NLP
practitioners. It affords philosophers a novel testing-ground for claims about
the nature of representation, and helps NLPers organise the large literature on
probing experiments, suggesting novel avenues for empirical research.
