  Recently a novel real-space RG algorithm was introduced, identifying the
relevant degrees of freedom of a system by maximizing an information-theoretic
quantity, the real-space mutual information (RSMI), with machine learning
methods. Motivated by this, we investigate the information theoretic properties
of coarse-graining procedures, for both translationally invariant and
disordered systems. We prove that a perfect RSMI coarse-graining does not
increase the range of interactions in the renormalized Hamiltonian, and, for
disordered systems, suppresses generation of correlations in the renormalized
disorder distribution, being in this sense optimal. We empirically verify decay
of those measures of complexity, as a function of information retained by the
RG, on the examples of arbitrary coarse-grainings of the clean and random Ising
chain. The results establish a direct and quantifiable connection between
properties of RG viewed as a compression scheme, and those of physical objects
i.e. Hamiltonians and disorder distributions. We also study the effect of
constraints on the number and type of coarse-grained degrees of freedom on a
generic RG procedure.
