  Neural network accelerator is a key enabler for the on-device AI inference,
for which energy efficiency is an important metric. The data-path energy,
including the computation energy and the data movement energy among the
arithmetic units, claims a significant part of the total accelerator energy. By
revisiting the basic physics of the arithmetic logic circuits, we show that the
data-path energy is highly correlated with the bit flips when streaming the
input operands into the arithmetic units, defined as the hamming distance of
the input operand matrices. Based on the insight, we propose a post-training
optimization algorithm and a hamming-distance-aware training algorithm to
co-design and co-optimize the accelerator and the network synergistically. The
experimental results based on post-layout simulation with MobileNetV2
demonstrate on average 2.85X data-path energy reduction and up to 8.51X
data-path energy reduction for certain layers.
