  Due to energy-efficiency requirements, computational systems are now being
implemented using noisy nanoscale semiconductor devices whose reliability
depends on energy consumed. We study circuit-level energy-reliability limits
for deep feedforward neural networks (multilayer perceptrons) built using such
devices, and en route also establish the same limits for formulas (boolean
tree-structured circuits). To obtain energy lower bounds, we extend Pippenger's
mutual information propagation technique for characterizing the complexity of
noisy circuits, since small circuit complexity need not imply low energy. Many
device technologies require all gates to have the same electrical operating
point; in circuits of such uniform gates, we show that the minimum energy
required to achieve any non-trivial reliability scales superlinearly with the
number of inputs. Circuits implemented in emerging device technologies like
spin electronics can, however, have gates operate at different electrical
points; in circuits of such heterogeneous gates, we show energy scaling can be
linear in the number of inputs. Building on our extended mutual information
propagation technique and using crucial insights from convex optimization
theory, we develop an algorithm to compute energy lower bounds for any given
boolean tree under heterogeneous gates. This algorithm runs in linear time in
number of gates, and is therefore practical for modern circuit design. As part
of our development we find a simple procedure for energy allocation across
circuit gates with different operating points and neural networks with
differently-operating layers.
