  In recent years Deep Learning reached significant results in many practical
problems, such as computer vision, natural language processing, speech
recognition and many others. For many years the main goal of the research was
to improve the quality of models, even if the complexity was impractically
high. However, for the production solutions, which often require real-time
work, the latency of the model plays a very important role. Current
state-of-the-art architectures are found with neural architecture search (NAS)
taking model complexity into account. However, designing of the search space
suitable for specific hardware is still a challenging task. To address this
problem we propose a measure of hardware efficiency of neural architecture
search space - matrix efficiency measure (MEM); a search space comprising of
hardware-efficient operations; a latency-aware scaling method; and ISyNet - a
set of architectures designed to be fast on the specialized neural processing
unit (NPU) hardware and accurate at the same time. We show the advantage of the
designed architectures for the NPU devices on ImageNet and the generalization
ability for the downstream classification and detection tasks.
