  Split computing has emerged as a recent paradigm for implementation of
DNN-based AI workloads, wherein a DNN model is split into two parts, one of
which is executed on a mobile/client device and the other on an edge-server (or
cloud). Data compression is applied to the intermediate tensor from the DNN
that needs to be transmitted, addressing the challenge of optimizing the
rate-accuracy-complexity trade-off. Existing split-computing approaches adopt
ML-based data compression, but require that the parameters of either the entire
DNN model, or a significant portion of it, be retrained for different
compression levels. This incurs a high computational and storage burden:
training a full DNN model from scratch is computationally demanding,
maintaining multiple copies of the DNN parameters increases storage
requirements, and switching the full set of weights during inference increases
memory bandwidth. In this paper, we present an approach that addresses all
these challenges. It involves the systematic design and training of bottleneck
units - simple, low-cost neural networks - that can be inserted at the point of
split. Our approach is remarkably lightweight, both during training and
inference, highly effective and achieves excellent rate-distortion performance
at a small fraction of the compute and storage overhead compared to existing
methods.
