  This paper presents PreVIous, a methodology to predict the performance of
convolutional neural networks (CNNs) in terms of throughput and energy
consumption on vision-enabled devices for the Internet of Things. CNNs
typically constitute a massive computational load for such devices, which are
characterized by scarce hardware resources to be shared among multiple
concurrent tasks. Therefore, it is critical to select the optimal CNN
architecture for a particular hardware platform according to prescribed
application requirements. However, the zoo of CNN models is already vast and
rapidly growing. To facilitate a suitable selection, we introduce a prediction
framework that allows to evaluate the performance of CNNs prior to their actual
implementation. The proposed methodology is based on PreVIousNet, a neural
network specifically designed to build accurate per-layer performance
predictive models. PreVIousNet incorporates the most usual parameters found in
state-of-the-art network architectures. The resulting predictive models for
inference time and energy have been tested against comprehensive
characterizations of seven well-known CNN models running on two different
software frameworks and two different embedded platforms. To the best of our
knowledge, this is the most extensive study in the literature concerning CNN
performance prediction on low-power low-cost devices. The average deviation
between predictions and real measurements is remarkably low, ranging from 3% to
10%. This means state-of-the-art modeling accuracy. As an additional asset, the
fine-grained a priori analysis provided by PreVIous could also be exploited by
neural architecture search engines.
