  The information bottleneck (IB) method offers an attractive framework for
understanding representation learning, however its applications are often
limited by its computational intractability. Analytical characterization of the
IB method is not only of practical interest, but it can also lead to new
insights into learning phenomena. Here we consider a generalized IB problem, in
which the mutual information in the original IB method is replaced by
correlation measures based on Renyi and Jeffreys divergences. We derive an
exact analytical IB solution for the case of Gaussian correlated variables. Our
analysis reveals a series of structural transitions, similar to those
previously observed in the original IB case. We find further that although
solving the original, Renyi and Jeffreys IB problems yields different
representations in general, the structural transitions occur at the same
critical tradeoff parameters, and the Renyi and Jeffreys IB solutions perform
well under the original IB objective. Our results suggest that formulating the
IB method with alternative correlation measures could offer a strategy for
obtaining an approximate solution to the original IB problem.
