  Zero-shot sim-to-real transfer of tasks with complex dynamics is a highly
challenging and unsolved problem. A number of solutions have been proposed in
recent years, but we have found that many works do not present a thorough
evaluation in the real world, or underplay the significant engineering effort
and task-specific fine tuning that is required to achieve the published
results. In this paper, we dive deeper into the sim-to-real transfer challenge,
investigate why this is such a difficult problem, and present objective
evaluations of a number of transfer methods across a range of real-world tasks.
Surprisingly, we found that a method which simply injects random forces into
the simulation performs just as well as more complex methods, such as those
which randomise the simulator's dynamics parameters, or adapt a policy online
using recurrent network architectures.
