  Deep neural networks are getting larger. Their implementation on edge and IoT
devices becomes more challenging and moved the community to design lighter
versions with similar performance. Standard automatic design tools such as
\emph{reinforcement learning} and \emph{evolutionary computing} fundamentally
rely on cheap evaluations of an objective function. In the neural network
design context, this objective is the accuracy after training, which is
expensive and time-consuming to evaluate. We automate the design of a light
deep neural network for image classification using the \emph{Mesh Adaptive
Direct Search}(MADS) algorithm, a mature derivative-free optimization method
that effectively accounts for the expensive blackbox nature of the objective
function to explore the design space, even in the presence of constraints.Our
tests show competitive compression rates with reduced numbers of trials.
