  Machine learning has changed the computing paradigm. Products today are built
with machine intelligence as a central attribute, and consumers are beginning
to expect near-human interaction with the appliances they use. However, much of
the deep learning revolution has been limited to the cloud. Recently, several
machine learning packages based on edge devices have been announced which aim
to offload the computing to the edges. However, little research has been done
to evaluate these packages on the edges, making it difficult for end users to
select an appropriate pair of software and hardware. In this paper, we make a
performance comparison of several state-of-the-art machine learning packages on
the edges, including TensorFlow, Caffe2, MXNet, PyTorch, and TensorFlow Lite.
We focus on evaluating the latency, memory footprint, and energy of these tools
with two popular types of neural networks on different edge devices. This
evaluation not only provides a reference to select appropriate combinations of
hardware and software packages for end users but also points out possible
future directions to optimize packages for developers.
