  As deep learning models become popular, there is a lot of need for deploying
them to diverse device environments. Because it is costly to develop and
optimize a neural network for every single environment, there is a line of
research to search neural networks for multiple target environments
efficiently. However, existing works for such a situation still suffer from
requiring many GPUs and expensive costs. Motivated by this, we propose a novel
neural network optimization framework named Bespoke for low-cost deployment.
Our framework searches for a lightweight model by replacing parts of an
original model with randomly selected alternatives, each of which comes from a
pretrained neural network or the original model. In the practical sense,
Bespoke has two significant merits. One is that it requires near zero cost for
designing the search space of neural networks. The other merit is that it
exploits the sub-networks of public pretrained neural networks, so the total
cost is minimal compared to the existing works. We conduct experiments
exploring Bespoke's the merits, and the results show that it finds efficient
models for multiple targets with meager cost.
