  We explore a few common models on how correlations affect information. The
main model considered is the Shannon mutual information $I(S:R_1,\cdots, R_i)$
over distributions with marginals $P_{S,R_i}$ fixed for each $i$, with the
analogy in which $S$ is the stimulus and $R_i$'s are neurons. We work out basic
models in details, using algebro-geometric tools to write down discriminants
that separate distributions with distinct qualitative behaviours in the
probability simplex into toric chambers and evaluate the volumes of them
algebraically. As a byproduct, we provide direct translation between a
decomposition of mutual information inspired by a series expansion and one from
partial information decomposition (PID) problems, characterising the
synergistic terms of the former. We hope this paper serves for communication
between communities especially mathematics and theoretical neuroscience on the
topic.
  KEYWORDS: information theory, algebraic statistics, mathematical
neuroscience, partial information decomposition
