  We apply a general theory describing the dynamics of supervised learning in
layered neural networks in the regime where the size p of the training set is
proportional to the number of inputs N, as developed in a previous paper, to
several choices of learning rules. In the case of (on-line and batch) Hebbian
learning, where a direct exact solution is possible, we show that our theory
provides exact results at any time in many different verifiable cases. For
non-Hebbian learning rules, such as Perceptron and AdaTron, we find very good
agreement between the predictions of our theory and numerical simulations.
Finally, we derive three approximation schemes aimed at eliminating the need to
solve a functional saddle-point equation at each time step, and assess their
performance. The simplest of these schemes leads to a fully explicit and
relatively simple non-linear diffusion equation for the joint field
distribution, which already describes the learning dynamics surprisingly well
over a wide range of parameters.
