  We introduce a fast and efficient convolutional neural network, ESPNet, for
semantic segmentation of high resolution images under resource constraints.
ESPNet is based on a new convolutional module, efficient spatial pyramid (ESP),
which is efficient in terms of computation, memory, and power. ESPNet is 22
times faster (on a standard GPU) and 180 times smaller than the
state-of-the-art semantic segmentation network PSPNet, while its category-wise
accuracy is only 8% less. We evaluated ESPNet on a variety of semantic
segmentation datasets including Cityscapes, PASCAL VOC, and a breast biopsy
whole slide image dataset. Under the same constraints on memory and
computation, ESPNet outperforms all the current efficient CNN networks such as
MobileNet, ShuffleNet, and ENet on both standard metrics and our newly
introduced performance metrics that measure efficiency on edge devices. Our
network can process high resolution images at a rate of 112 and 9 frames per
second on a standard GPU and edge device, respectively.
