  Nowadays most research in visual recognition using Convolutional Neural
Networks (CNNs) follows the "deeper model with deeper confidence" belief to
gain a higher recognition accuracy. At the same time, deeper model brings
heavier computation. On the other hand, for a large chunk of recognition
challenges, a system can classify images correctly using simple models or
so-called shallow networks. Moreover, the implementation of CNNs faces with the
size, weight, and energy constraints on the embedded devices. In this paper, we
implement the adaptive switching between shallow and deep networks to reach the
highest throughput on a resource-constrained MPSoC with CPU and FPGA. To this
end, we develop and present a novel architecture for the CNNs where a gate
makes the decision whether using the deeper model is beneficial or not. Due to
resource limitation on FPGA, the idea of partial reconfiguration has been used
to accommodate deep CNNs on the FPGA resources. We report experimental results
on CIFAR-10, CIFAR-100, and SVHN datasets to validate our approach. Using
confidence metric as the decision making factor, only 69.8%, 71.8%, and 43.8%
of the computation in the deepest network is done for CIFAR-10, CIFAR-100, and
SVHN while it can maintain the desired accuracy with the throughput of around
400 images per second for SVHN dataset.
