  We calculate the mutual information (MI) of a two-layered neural network with
noiseless, continuous inputs and binary, stochastic outputs under several
assumptions on the synaptic efficiencies. The interesting regime corresponds to
the limit where the number of both input and output units is large but their
ratio is kept fixed at a value $\alpha$. We first present a solution for the MI
using the replica technique with a replica symmetric (RS) ansatz. Then we find
an exact solution for this quantity valid in a neighborhood of $\alpha = 0$. An
analysis of this solution shows that the system must have a phase transition at
some finite value of $\alpha$. This transition shows a singularity in the third
derivative of the MI. As the RS solution turns out to be infinitely
differentiable, it could be regarded as a smooth approximation to the MI. This
is checked numerically in the validity domain of the exact solution.
