  The control and guidance of multi-robots (swarm) is a non-trivial problem due
to the complexity inherent in the coupled interaction among the group. Whether
the swarm is cooperative or non-cooperative, lessons can be learnt from
sheepdogs herding sheep. Biomimicry of shepherding offers computational methods
for swarm control with the potential to generalize and scale in different
environments. However, learning to shepherd is complex due to the large search
space that a machine learner is faced with. We present a deep hierarchical
reinforcement learning approach for shepherding, whereby an unmanned aerial
vehicle (UAV) learns to act as an aerial sheepdog to control and guide a swarm
of unmanned ground vehicles (UGVs). The approach extends our previous work on
machine education to decompose the search space into a hierarchically organized
curriculum. Each lesson in the curriculum is learnt by a deep reinforcement
learning model. The hierarchy is formed by fusing the outputs of the model. The
approach is demonstrated first in a high-fidelity robotic-operating-system
(ROS)-based simulation environment, then with physical UGVs and a UAV in an
in-door testing facility. We investigate the ability of the method to
generalize as the models move from simulation to the real-world and as the
models move from one scale to another.
