  The analysis of complex physical systems hinges on the ability to extract the
relevant degrees of freedom from among the many others. Though much hope is
placed in machine learning, it also brings challenges, chief of which is
interpretability. It is often unclear what relation, if any, the architecture-
and training-dependent learned "relevant" features bear to standard objects of
physical theory. Here we report on theoretical results which may help to
systematically address this issue: we establish equivalence between the
information-theoretic notion of relevance defined in the Information Bottleneck
(IB) formalism of compression theory, and the field-theoretic relevance of the
Renormalization Group. We show analytically that for statistical physical
systems described by a field theory the "relevant" degrees of freedom found
using IB compression indeed correspond to operators with the lowest scaling
dimensions. We confirm our field theoretic predictions numerically. We study
dependence of the IB solutions on the physical symmetries of the data. Our
findings provide a dictionary connecting two distinct theoretical toolboxes,
and an example of constructively incorporating physical interpretability in
applications of deep learning in physics.
