  We introduce the problem of model-extraction attacks in cyber-physical
systems in which an attacker attempts to estimate (or extract) the feedback
controller of the system. Extracting (or estimating) the controller provides an
unmatched edge to attackers since it allows them to predict the future control
actions of the system and plan their attack accordingly. Hence, it is important
to understand the ability of the attackers to perform such an attack. In this
paper, we focus on the setting when a Deep Neural Network (DNN) controller is
trained using Reinforcement Learning (RL) algorithms and is used to control a
stochastic system. We play the role of the attacker that aims to estimate such
an unknown DNN controller, and we propose a two-phase algorithm. In the first
phase, also called the offline phase, the attacker uses side-channel
information about the RL-reward function and the system dynamics to identify a
set of candidate estimates of the unknown DNN. In the second phase, also called
the online phase, the attacker observes the behavior of the unknown DNN and
uses these observations to shortlist the set of final policy estimates. We
provide theoretical analysis of the error between the unknown DNN and the
estimated one. We also provide numerical results showing the effectiveness of
the proposed algorithm.
