  Cloud-based machine learning services (CMLS) enable organizations to take
advantage of advanced models that are pre-trained on large quantities of data.
The main shortcoming of using these services, however, is the difficulty of
keeping the transmitted data private and secure. Asymmetric encryption requires
the data to be decrypted in the cloud, while Homomorphic encryption is often
too slow and difficult to implement. We propose One Way Scrambling by
Deconvolution (OWSD), a deconvolution-based scrambling framework that offers
the advantages of Homomorphic encryption at a fraction of the computational
overhead. Extensive evaluation on multiple image datasets demonstrates OWSD's
ability to achieve near-perfect classification performance when the output
vector of the CMLS is sufficiently large. Additionally, we provide empirical
analysis of the robustness of our approach.
