  Understanding the gap between simulation and reality is critical for
reinforcement learning with legged robots, which are largely trained in
simulation. However, recent work has resulted in sometimes conflicting
conclusions with regard to which factors are important for success, including
the role of dynamics randomization. In this paper, we aim to provide clarity
and understanding on the role of dynamics randomization in learning robust
locomotion policies for the Laikago quadruped robot. Surprisingly, in contrast
to prior work with the same robot model, we find that direct sim-to-real
transfer is possible without dynamics randomization or on-robot adaptation
schemes. We conduct extensive ablation studies in a sim-to-sim setting to
understand the key issues underlying successful policy transfer, including
other design decisions that can impact policy robustness. We further ground our
conclusions via sim-to-real experiments with various gaits, speeds, and
stepping frequencies. Additional Details:
https://www.pair.toronto.edu/understanding-dr/.
