  Sim-to-real transfer trains RL agents in the simulated environments and then
deploys them in the real world. Sim-to-real transfer has been widely used in
practice because it is often cheaper, safer and much faster to collect samples
in simulation than in the real world. Despite the empirical success of the
sim-to-real transfer, its theoretical foundation is much less understood. In
this paper, we study the sim-to-real transfer in continuous domain with partial
observations, where the simulated environments and real-world environments are
modeled by linear quadratic Gaussian (LQG) systems. We show that a popular
robust adversarial training algorithm is capable of learning a policy from the
simulated environment that is competitive to the optimal policy in the
real-world environment. To achieve our results, we design a new algorithm for
infinite-horizon average-cost LQGs and establish a regret bound that depends on
the intrinsic complexity of the model class. Our algorithm crucially relies on
a novel history clipping scheme, which might be of independent interest.
