  We present a three-step method to perform system identification and optimal
control of non-linear systems. Our approach is mainly data driven and does not
require active excitation of the system to perform system identification. In
particular, it is designed for systems for which only historical data under
closed-loop control are available and where historical control commands exhibit
low variability. In a first step, simple simulation models of the system are
built and run under various conditions. In a second step, a neural network
architecture is extensively trained on the simulation outputs to learn the
system physics, and retrained with historical data from the real system with
stopping rules. These constraints avoid overfitting that arise by fitting
closed-loop controlled systems. By doing so, we obtain one (or many) system
model(s), represented by this architecture, and whose behaviour can be chosen
to match more or less the real system. Finally, state-of-the-art reinforcement
learning with a variant of domain randomization and distributed learning is
used for optimal control of the system. We first illustrate the model
identification strategy with a simple example, the pendulum with external
torque. We then apply our method to model and optimize the control of a large
building facility located in Switzerland. Simulation results demonstrate that
this approach generates stable functional controllers which outperform on
comfort and energy benchmark rule-based controllers.
