  In recent years, domain randomization over dynamics parameters has gained a
lot of traction as a method for sim-to-real transfer of reinforcement learning
policies in robotic manipulation; however, finding optimal randomization
distributions can be difficult. In this paper, we introduce DROPO, a novel
method for estimating domain randomization distributions for safe sim-to-real
transfer. Unlike prior work, DROPO only requires a limited, precollected
offline dataset of trajectories, and explicitly models parameter uncertainty to
match real data using a likelihood-based approach. We demonstrate that DROPO is
capable of recovering dynamic parameter distributions in simulation and finding
a distribution capable of compensating for an unmodeled phenomenon. We also
evaluate the method in two zero-shot sim-to-real transfer scenarios, showing
successful domain transfer and improved performance over prior methods.
