  Reproducibility in reinforcement learning is challenging: uncontrolled
stochasticity from many sources, such as the learning algorithm, the learned
policy, and the environment itself have led researchers to report the
performance of learned agents using aggregate metrics of performance over
multiple random seeds for a single environment. Unfortunately, there are still
pernicious sources of variability in reinforcement learning agents that make
reporting common summary statistics an unsound metric for performance. Our
experiments demonstrate the variability of common agents used in the popular
OpenAI Baselines repository. We make the case for reporting post-training agent
performance as a distribution, rather than a point estimate.
