  Learning algorithms, like Quality-Diversity (QD), can be used to acquire
repertoires of diverse robotics skills. This learning is commonly done via
computer simulation due to the large number of evaluations required. However,
training in a virtual environment generates a gap between simulation and
reality. Here, we build upon the Reset-Free QD (RF-QD) algorithm to learn
controllers directly on a physical robot. This method uses a dynamics model,
learned from interactions between the robot and the environment, to predict the
robot's behaviour and improve sample efficiency. A behaviour selection policy
filters out uninteresting or unsafe policies predicted by the model. RF-QD also
includes a recovery policy that returns the robot to a safe zone when it has
walked outside of it, allowing continuous learning. We demonstrate that our
method enables a physical quadruped robot to learn a repertoire of behaviours
in two hours without human supervision. We successfully test the solution
repertoire using a maze navigation task. Finally, we compare our approach to
the MAP-Elites algorithm. We show that dynamics awareness and a recovery policy
are required for training on a physical robot for optimal archive generation.
Video available at https://youtu.be/BgGNvIsRh7Q
