  We study the capacity of \emph{sign} perceptrons neural networks (SPNN) and
particularly focus on 1-hidden layer \emph{treelike committee machine} (TCM)
architectures. Similarly to what happens in the case of a single perceptron
neuron, it turns out that, in a statistical sense, the capacity of a
corresponding multilayered network architecture consisting of multiple
\emph{sign} perceptrons also undergoes the so-called phase transition (PT)
phenomenon. This means: (i) for certain range of system parameters (size of
data, number of neurons), the network can be properly trained to accurately
memorize \emph{all} elements of the input dataset; and (ii) outside the region
such a training does not exist. Clearly, determining the corresponding phase
transition curve that separates these regions is an extraordinary task and
among the most fundamental questions related to the performance of any network.
Utilizing powerful mathematical engine called Random Duality Theory (RDT), we
establish a generic framework for determining the upper bounds on the 1-hidden
layer TCM SPNN capacity. Moreover, we do so for \emph{any} given (odd) number
of neurons. We further show that the obtained results \emph{exactly} match the
replica symmetry predictions of \cite{EKTVZ92,BHS92}, thereby proving that the
statistical physics based results are not only nice estimates but also
mathematically rigorous bounds as well. Moreover, for $d\leq 5$, we obtain the
capacity values that improve on the best known rigorous ones of
\cite{MitchDurb89}, thereby establishing a first, mathematically rigorous,
progress in well over 30 years.
