  Edge computing has emerged as a popular paradigm for supporting mobile and
IoT applications with low latency or high bandwidth needs. The attractiveness
of edge computing has been further enhanced due to the recent availability of
special-purpose hardware to accelerate specific compute tasks, such as deep
learning inference, on edge nodes. In this paper, we experimentally compare the
benefits and limitations of using specialized edge systems, built using edge
accelerators, to more traditional forms of edge and cloud computing. Our
experimental study using edge-based AI workloads shows that today's edge
accelerators can provide comparable, and in many cases better, performance,
when normalized for power or cost, than traditional edge and cloud servers.
They also provide latency and bandwidth benefits for split processing, across
and within tiers, when using model compression or model splitting, but require
dynamic methods to determine the optimal split across tiers. We find that edge
accelerators can support varying degrees of concurrency for multi-tenant
inference applications, but lack isolation mechanisms necessary for edge cloud
multi-tenant hosting.
