  There is increasing demand for specialized hardware for training deep neural
networks, both in edge/IoT environments and in high-performance computing
systems. The design space of such hardware is very large due to the wide range
of processing architectures, deep neural network configurations, and dataflow
options. This makes developing deep neural network processors quite complex,
especially for training. We present TRIM, an infrastructure to help hardware
architects explore the design space of deep neural network accelerators for
both inference and training in the early design stages. The model evaluates at
the whole network level, considering both inter-layer and intra-layer
activities. Given applications, essential hardware specifications, and a design
goal, TRIM can quickly explore different hardware design options, select the
optimal dataflow and guide new hardware architecture design. We validated TRIM
with FPGA-based implementation of deep neural network accelerators and
ASIC-based architectures. We also show how to use TRIM to explore the design
space through several case studies. TRIM is a powerful tool to help architects
evaluate different hardware choices to develop efficient inference and training
architecture design.
