  To achieve high accuracy, convolutional neural networks (CNNs) are
increasingly growing in complexity and diversity in layer types and topologies.
This makes it very challenging to efficiently deploy such networks on custom
processor architectures for resource-scarce edge devices. Existing mapping
exploration frameworks enable searching for the optimal execution schedules or
hardware mappings of individual network layers, by optimizing each layer's
spatial (dataflow parallelization) and temporal unrolling (execution order).
However, these tools fail to take into account the overhead of supporting
different unrolling schemes within a common hardware architecture. Using a
fixed unrolling scheme across all layers is also not ideal, as this misses
significant opportunities for energy and latency savings from optimizing the
mapping of diverse layer types. A balanced approach assesses the right amount
of mapping flexibility needed across target neural networks, while taking into
account the overhead to support multiple unrollings. This paper, therefore,
presents COAC, a cross-layer design space exploration and mapping framework to
optimize the flexibility of neural processing architectures by balancing
configurability overhead against resulting energy and latency savings for
end-to-end inference. COAC does not only provide a systematical analysis of the
architectural overhead in function of the supported spatial unrollings, but
also builds an automated flow to find the best unrolling combination(s) for
efficient end-to-end inference with limited hardware overhead. Results
demonstrate that architectures with carefully optimized flexibility can achieve
up to 38% EDP (energy-delay-product) savings for a set of six neural networks
at the expense of a relative area increase of 9.5%.
