  Deep Neural Networks (DNNs) may be partitioned across the edge and the cloud
to improve the performance efficiency of inference. DNN partitions are
determined based on operational conditions such as network speed. When
operational conditions change DNNs will need to be repartitioned to maintain
the overall performance. However, repartitioning using existing approaches,
such as Pause and Resume, will incur a service downtime on the edge. This paper
presents the NEUKONFIG framework that identifies the service downtime incurred
when repartitioning DNNs and proposes approaches for reducing edge service
downtime. The proposed approaches are based on 'Dynamic Switching' in which,
when the network speed changes and given an existing edge-cloud pipeline, a new
edge-cloud pipeline is initialised with new DNN partitions. Incoming inference
requests are switched to the new pipeline for processing data. Two dynamic
switching scenarios are considered: when a second edge-cloud pipeline is always
running and when a second pipeline is only initialised when the network speed
changes. Experimental studies are carried out on a lab-based testbed to
demonstrate that Dynamic Switching reduces the downtime by at least an order of
magnitude when compared to a baseline using Pause and Resume that has a
downtime of 6 seconds. A trade-off in the edge service downtime and memory
required is noted. The Dynamic Switching approach that requires the same amount
of memory as the baseline reduces the edge service downtime to 0.6 seconds and
to less than 1 millisecond in the best case when twice the amount of memory as
the baseline is available.
