  Understanding which concepts models can and cannot represent has been
fundamental to many tasks: from effective and responsible use of models to
detecting out of distribution data. We introduce Gaussian process probes (GPP),
a unified and simple framework for probing and measuring uncertainty about
concepts represented by models. As a Bayesian extension of linear probing
methods, GPP asks what kind of distribution over classifiers (of concepts) is
induced by the model. This distribution can be used to measure both what the
model represents and how confident the probe is about what the model
represents. GPP can be applied to any pre-trained model with vector
representations of inputs (e.g., activations). It does not require access to
training data, gradients, or the architecture. We validate GPP on datasets
containing both synthetic and real images. Our experiments show it can (1)
probe a model's representations of concepts even with a very small number of
examples, (2) accurately measure both epistemic uncertainty (how confident the
probe is) and aleatory uncertainty (how fuzzy the concepts are to the model),
and (3) detect out of distribution data using those uncertainty measures as
well as classic methods do. By using Gaussian processes to expand what probing
can offer, GPP provides a data-efficient, versatile and uncertainty-aware tool
for understanding and evaluating the capabilities of machine learning models.
