  With the edge computing becoming an increasingly adopted concept in system
architectures, it is expected its utilization will be additionally heightened
when combined with deep learning (DL) techniques. The idea behind integrating
demanding processing algorithms in Internet of Things (IoT) and edge devices,
such as Deep Neural Network (DNN), has in large measure benefited from the
development of edge computing hardware, as well as from adapting the algorithms
for use in resource constrained IoT devices. Surprisingly, there are no models
yet to optimally place and use machine learning in edge computing. In this
paper, we propose the first model of optimal placement of Deep Neural Network
(DNN) Placement and Inference in edge computing. We present a mathematical
formulation to the DNN Model Variant Selection and Placement (MVSP) problem
considering the inference latency of different model-variants, communication
latency between nodes, and utilization cost of edge computing nodes. We
evaluate our model numerically, and show that for low load increasing model
co-location decreases the average latency by 33% of millisecond-scale per
request, and for high load, by 21%.
