  We formulate and analyze the compound information bottleneck programming. In
this problem, a Markov chain $ \mathsf{X} \rightarrow \mathsf{Y} \rightarrow
\mathsf{Z} $ is assumed with fixed marginal distributions
$\mathsf{P}_{\mathsf{X}}$ and $\mathsf{P}_{\mathsf{Y}}$, and the mutual
information between $ \mathsf{X} $ and $ \mathsf{Z} $ is sought to be maximized
over the choice of conditional probability of $\mathsf{Z}$ given $\mathsf{Y}$
from a given class, under the \textit{worst choice} of the joint probability of
the pair $(\mathsf{X},\mathsf{Y})$ from a different class. We consider several
classes based on extremes of: mutual information; minimal correlation; total
variation; and the relative entropy class. We provide values, bounds, and
various characterizations for specific instances of this problem: the binary
symmetric case, the scalar Gaussian case, the vector Gaussian case and the
symmetric modulo-additive case. Finally, for the general case, we propose a
Blahut-Arimoto type of alternating iterations algorithm to find a consistent
solution to this problem.
