  As we aim to control complex systems, use of a simulator in model-based
reinforcement learning is becoming more common. However, it has been
challenging to overcome the Reality Gap, which comes from nonlinear model bias
and susceptibility to disturbance. To address these problems, we propose a
novel algorithm that combines data-driven system identification approach
(Gaussian Process) with a Differential-Dynamic-Programming-based robust optimal
control method (Iterative Linear Quadratic Control). Our algorithm uses the
simulator's model as the mean function for a Gaussian Process and learns only
the difference between the simulator's prediction and actual observations,
making it a natural hybrid of simulation and real-world observation. We show
that our approach quickly corrects incorrect models, comes up with robust
optimal controllers, and transfers its acquired model knowledge to new tasks
efficiently.
