  The entropy of a pair of random variables is commonly depicted using a Venn
diagram. This representation is potentially misleading, however, since the
multivariate mutual information can be negative. This paper presents new
measures of multivariate information content that can be accurately depicted
using Venn diagrams for any number of random variables. These measures
complement the existing measures of multivariate mutual information and are
constructed by considering the algebraic structure of information sharing. It
is shown that the distinct ways in which a set of marginal observers can share
their information with a non-observing third party corresponds to the elements
of a free distributive lattice. The redundancy lattice from partial information
decomposition is then subsequently and independently derived by combining the
algebraic structures of joint and shared information content.
