  Power consumption is a major obstacle in the deployment of deep neural
networks (DNNs) on end devices. Existing approaches for reducing power
consumption rely on quite general principles, including avoidance of
multiplication operations and aggressive quantization of weights and
activations. However, these methods do not take into account the precise power
consumed by each module in the network, and are therefore not optimal. In this
paper we develop accurate power consumption models for all arithmetic
operations in the DNN, under various working conditions. We reveal several
important factors that have been overlooked to date. Based on our analysis, we
present PANN (power-aware neural network), a simple approach for approximating
any full-precision network by a low-power fixed-precision variant. Our method
can be applied to a pre-trained network, and can also be used during training
to achieve improved performance. In contrast to previous methods, PANN incurs
only a minor degradation in accuracy w.r.t. the full-precision version of the
network, even when working at the power-budget of a 2-bit quantized variant. In
addition, our scheme enables to seamlessly traverse the power-accuracy
trade-off at deployment time, which is a major advantage over existing
quantization methods that are constrained to specific bit widths.
