  Designing deep learning-based solutions is becoming a race for training
deeper models with a greater number of layers. While a large-size deeper model
could provide competitive accuracy, it creates a lot of logistical challenges
and unreasonable resource requirements during development and deployment. This
has been one of the key reasons for deep learning models not being excessively
used in various production environments, especially in edge devices. There is
an immediate requirement for optimizing and compressing these deep learning
models, to enable on-device intelligence. In this research, we introduce a
black-box framework, Deeplite Neutrino for production-ready optimization of
deep learning models. The framework provides an easy mechanism for the
end-users to provide constraints such as a tolerable drop in accuracy or target
size of the optimized models, to guide the whole optimization process. The
framework is easy to include in an existing production pipeline and is
available as a Python Package, supporting PyTorch and Tensorflow libraries. The
optimization performance of the framework is shown across multiple benchmark
datasets and popular deep learning models. Further, the framework is currently
used in production and the results and testimonials from several clients are
summarized.
