  New data processing pipelines and novel network architectures increasingly
drive the success of deep learning. In consequence, the industry considers
top-performing architectures as intellectual property and devotes considerable
computational resources to discovering such architectures through neural
architecture search (NAS). This provides an incentive for adversaries to steal
these novel architectures; when used in the cloud, to provide Machine Learning
as a Service, the adversaries also have an opportunity to reconstruct the
architectures by exploiting a range of hardware side channels. However, it is
challenging to reconstruct novel architectures and pipelines without knowing
the computational graph (e.g., the layers, branches or skip connections), the
architectural parameters (e.g., the number of filters in a convolutional layer)
or the specific pre-processing steps (e.g. embeddings). In this paper, we
design an algorithm that reconstructs the key components of a novel deep
learning system by exploiting a small amount of information leakage from a
cache side-channel attack, Flush+Reload. We use Flush+Reload to infer the trace
of computations and the timing for each computation. Our algorithm then
generates candidate computational graphs from the trace and eliminates
incompatible candidates through a parameter estimation process. We implement
our algorithm in PyTorch and Tensorflow. We demonstrate experimentally that we
can reconstruct MalConv, a novel data pre-processing pipeline for malware
detection, and ProxylessNAS- CPU, a novel network architecture for the ImageNet
classification optimized to run on CPUs, without knowing the architecture
family. In both cases, we achieve 0% error. These results suggest hardware side
channels are a practical attack vector against MLaaS, and more efforts should
be devoted to understanding their impact on the security of deep learning
systems.
