  We study the problem of efficient compression of a stochastic source of
probability distributions. It can be viewed as a generalization of Shannon's
source coding problem. It has relation to the theory of common randomness, as
well as to channel coding and rate--distortion theory: in the first two
subjects ``inverses'' to established coding theorems can be derived, yielding a
new approach to proving converse theorems, in the third we find a new proof of
Shannon's rate--distortion theorem.
  After reviewing the known lower bound for the optimal compression rate, we
present a number of approaches to achieve it by code constructions. Our main
results are: a better understanding of the known lower bounds on the
compression rate by means of a strong version of this statement, a review of a
construction achieving the lower bound by using common randomness which we
complement by showing the optimal use of the latter within a class of
protocols. Then we review another approach, not dependent on common randomness,
to minimizing the compression rate, providing some insight into its
combinatorial structure, and suggesting an algorithm to optimize it.
  The second part of the paper is concerned with the generalization of the
problem to quantum information theory: the compression of mixed quantum states.
Here, after reviewing the known lower bound we contribute a strong version of
it, and discuss the relation of the problem to other issues in quantum
information theory.
