  Recently, reinforcement learning (RL) algorithms have demonstrated remarkable
success in learning complicated behaviors from minimally processed input.
However, most of this success is limited to simulation. While there are
promising successes in applying RL algorithms directly on real systems, their
performance on more complex systems remains bottle-necked by the relative data
inefficiency of RL algorithms. Domain randomization is a promising direction of
research that has demonstrated impressive results using RL algorithms to
control real robots. At a high level, domain randomization works by training a
policy on a distribution of environmental conditions in simulation. If the
environments are diverse enough, then the policy trained on this distribution
will plausibly generalize to the real world. A human-specified design choice in
domain randomization is the form and parameters of the distribution of
simulated environments. It is unclear how to the best pick the form and
parameters of this distribution and prior work uses hand-tuned distributions.
This extended abstract demonstrates that the choice of the distribution plays a
major role in the performance of the trained policies in the real world and
that the parameter of this distribution can be optimized to maximize the
performance of the trained policies in the real world
