  We consider the problem of designing models to leverage a recently introduced
approximate model averaging technique called dropout. We define a simple new
model called maxout (so named because its output is the max of a set of inputs,
and because it is a natural companion to dropout) designed to both facilitate
optimization by dropout and improve the accuracy of dropout's fast approximate
model averaging technique. We empirically verify that the model successfully
accomplishes both of these tasks. We use maxout and dropout to demonstrate
state of the art classification performance on four benchmark datasets: MNIST,
CIFAR-10, CIFAR-100, and SVHN.
