  While the utility of well-chosen abstractions for understanding and
predicting the behaviour of complex systems is well appreciated, precisely what
an abstraction $\textit{is}$ has so far has largely eluded mathematical
formalization. In this paper, we aim to set out a mathematical theory of
abstraction. We provide a precise characterisation of what an abstraction is
and, perhaps more importantly, suggest how abstractions can be learnt directly
from data both for static datasets and for dynamical systems. We define an
abstraction to be a small set of `summaries' of a system which can be used to
answer a set of queries about the system or its behaviour. The difference
between the ground truth behaviour of the system on the queries and the
behaviour of the system predicted only by the abstraction provides a measure of
the `leakiness' of the abstraction which can be used as a loss function to
directly learn abstractions from data. Our approach can be considered a
generalization of classical statistics where we are not interested in
reconstructing `the data' in full, but are instead only concerned with
answering a set of arbitrary queries about the data. While highly theoretical,
our results have deep implications for statistical inference and machine
learning and could be used to develop explicit methods for learning precise
kinds of abstractions directly from data.
