  Equilibrium statistical physics is applied to layered neural networks with
differentiable activation functions. A first analysis of off-line learning in
soft-committee machines with a finite number (K) of hidden units learning a
perfectly matching rule is performed. Our results are exact in the limit of
high training temperatures. For K=2 we find a second order phase transition
from unspecialized to specialized student configurations at a critical size P
of the training set, whereas for K > 2 the transition is first order. Monte
Carlo simulations indicate that our results are also valid for moderately low
temperatures qualitatively. The limit K to infinity can be performed
analytically, the transition occurs after presenting on the order of N K
examples. However, an unspecialized metastable state persists up to P= O (N
K^2).
