  Legged locomotion is widespread in nature and has inspired the design of
current robots. The controller of these legged robots is often realized as one
centralized instance. However, in nature, control of movement happens in a
hierarchical and decentralized fashion. Introducing these biological design
principles into robotic control systems has motivated this work. We tackle the
question whether decentralized and hierarchical control is beneficial for
legged robots and present a novel decentral, hierarchical architecture to
control a simulated legged agent. Three different tasks varying in complexity
are designed to benchmark five architectures (centralized, decentralized,
hierarchical and two different combinations of hierarchical decentralized
architectures). The results demonstrate that decentralizing the different
levels of the hierarchical architectures facilitates learning of the agent,
ensures more energy efficient movements as well as robustness towards new
unseen environments. Furthermore, this comparison sheds light on the importance
of modularity in hierarchical architectures to solve complex goal-directed
tasks. We provide an open-source code implementation of our architecture
(https://github.com/wzaielamri/hddrl).
