  A efficient incremental learning algorithm for classification tasks, called
NetLines, well adapted for both binary and real-valued input patterns is
presented. It generates small compact feedforward neural networks with one
hidden layer of binary units and binary output units. A convergence theorem
ensures that solutions with a finite number of hidden units exist for both
binary and real-valued input patterns. An implementation for problems with more
than two classes, valid for any binary classifier, is proposed. The
generalization error and the size of the resulting networks are compared to the
best published results on well-known classification benchmarks. Early stopping
is shown to decrease overfitting, without improving the generalization
performance.
