  This tutorial paper focuses on the variants of the bottleneck problem taking
an information theoretic perspective and discusses practical methods to solve
it, as well as its connection to coding and learning aspects. The intimate
connections of this setting to remote source-coding under logarithmic loss
distortion measure, information combining, common reconstruction, the
Wyner-Ahlswede-Korner problem, the efficiency of investment information, as
well as, generalization, variational inference, representation learning,
autoencoders, and others are highlighted. We discuss its extension to the
distributed information bottleneck problem with emphasis on the Gaussian model
and highlight the basic connections to the uplink Cloud Radio Access Networks
(CRAN) with oblivious processing. For this model, the optimal trade-offs
between relevance (i.e., information) and complexity (i.e., rates) in the
discrete and vector Gaussian frameworks is determined. In the concluding
outlook, some interesting problems are mentioned such as the characterization
of the optimal inputs ("features") distributions under power limitations
maximizing the "relevance" for the Gaussian information bottleneck, under
"complexity" constraints.
