  We present a neural network architecture and training method designed to
enable very rapid training and low implementation complexity. Due to its
training speed and very few tunable parameters, the method has strong potential
for applications requiring frequent retraining or online training. The approach
is characterized by (a) convolutional filters based on biologically inspired
visual processing filters, (b) randomly-valued classifier-stage input weights,
(c) use of least squares regression to train the classifier output weights in a
single batch, and (d) linear classifier-stage output units. We demonstrate the
efficacy of the method by applying it to image classification. Our results
match existing state-of-the-art results on the MNIST (0.37% error) and
NORB-small (2.2% error) image classification databases, but with very fast
training times compared to standard deep network approaches. The network's
performance on the Google Street View House Number (SVHN) (4% error) database
is also competitive with state-of-the art methods.
