  Existing side-channel analysis techniques require a leakage model, in the
form of a prior knowledge or a set of training data, to establish a
relationship between the secret data and the measurements. We introduce
side-channel analysis with reinforcement learning (SCARL) capable of extracting
data-dependent features of the measurements in an unsupervised learning
approach without requiring a prior knowledge on the leakage model. SCARL
consists of an auto-encoder to encode the information of power measurements
into an internal representation, and a reinforcement learning algorithm to
extract information about the secret data. We employ a reinforcement learning
algorithm with actor-critic networks, to identify the proper leakage model that
results in maximum inter-cluster separation of the auto-encoder representation.
SCARL assumes that the lower order components of a generic non-linear leakage
model have larger contribution to the leakage of sensitive data. On a
lightweight implementation of the Ascon authenticated cipher on the Artix-7
FPGA, SCARL is able to recover the secret key using 24K power traces during the
key insertion, or Initialization Stage, of the cipher. We also demonstrate that
classical techniques such as DPA and CPA fail to identify the correct key using
traditional linear leakage models and more than 40K power traces.
